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PREFACE 


The  Departwnt  of  Defense  Procureaent  Research 
Coordinating  Council  is  proud  to  have  sponsored  the  6th 
Annual  DoD  Procureoent  Research  Symposium.  These  proceed- 
ings reflect  three  full  days  of  brisk  activity  **  of  50 
different  speakers,  lively  discussions,  and  general  profes- 
sional growth,  not  to  nention  good  food,  fine  coapaxiy,  and 
outstanding  Uest  Point  hospitality.  We  are  all  indebted  to 
^e  Conaandant  and  the  USMA  Staff  for  their  efforts. 


We,  of  course,  want  to  capture  part  of  that  activity  and 
share  it  with  those  in  the  acquisition  and  research  coanittees 
who  could  not  attend.  I believe  that  ouch  of  the  work  described 
will  endure  and  hope  that  this  document  %rill  become  a valuable 
part,  of  the  library  of  both  the  procurement  researchers  and 
the  procurement  managers. 


The  Council  is  indebted  to  the  speakers  and  participants 
who  made  this  meeting  a success.  We  also  owe  special  thanks 
to  the  Amy  Procurement  Research  Office  (AFRO)  who  developed  the 
symposium  and  gathered  these  proceedings  and  to  our  host,  the 
United  States  Military  Acadeny. 
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4SEMILLER 

' Chairman 

DoD  Procurement  Research 
Coordinating  Council 
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yaCONING  ADDRESS 


VJ 

( ) LTG  Andrew  J.  Goodpester 

^ SupeHntendent.  US  milUry  Acidav 
West  Point 


Kiy  I extend  a very  warn  and  cordfal  wHcok  to  all  of  you  to  the  United  States  milUry 
Academy.  As  you  krow,  I do  this  on  the  strength  of  ay  having  been  here  for  about  one 
I 1 week  lyself.  It's  a great  deal  of  p<easure  to  welcowe  you  and  your  ladles  to  this  6th 
V-y  Annual  Departnent  of  Defense  Procurewent  Research  Sywposlia.  I think  you  will  agree 
that  you  can  hardly  have  chosen  a wore  beautiful  and  historical  location  or  a wore 
delightful  day.  As  I was  saying  as  we  walked  over,  we  don't  like  to  dispense  too  itpy 
of  these.  We  like  to  save  a few  of  thew  becatise  there  cooes  a tine  when  we  know  that 
we'll  need  theo. 

Tour  concern  Is  with  procureaent.  Itere  specifically  with  procureoent  research.  And 
( ) even  wore  specifically,  translating  procureaiefit  research  into  action.  It's  a concern  that 

we  all  share,  each  In  his  or  her  own  way. 

GovemKnt  procurewnt,  siwply  stated  I suppose.  Is  the  act  of  obtaining,  normally 
ff»  some  conerclal  source,  the  necessary  supplies,  equipment  or  material  to  support 
all  the  Department  of  Defense  activities.  As  we  know  there  are  many  activities,  ranging 
from  the  highest  echelon  of  conmnd  to  the  lowest  user  level  which  must  be  supported 
as  efficiently  and  as  economically  as  possible,  duHng  both  peace  and  war.  That's  a great 
charge  and  a great  responsibility.  And  particularly  as  the  boundaries  of  technology  and  ^ 
weapon  systems  and  Intraspace  travel  are  pushed  out  further,  the  method  of  procuring  the 
overall  support  for  these  programs  and  the  system*  must  stay  abreast.  That  Is  a heavy  charge, 
because  aqy  program  moves  only  as  fast  as  Its  least  developed  eleaient.  The  method  of 
^talnlng  toUl  system  support  must  therefore  not  lag  behind  or  hold  up  the  rapid  progress 
In  other  fields  of  technology. 

Although  I don't  claim  to  be  an  expert  In  your  particular  field,  I would  suppose 
[ that  research  In  your  field.  In  Its  basic  form.  Is  an  ongoing  challenge  of  seeking  the  proper 
procurement  method,  idiether  It  be  new  or  old,  or  a mixture  of  the  two  required  to  meet 
the  needs  of  today  using  the  strengths  of  the  past  with  flexibility  for  tomorrow.  The 
challenge  to  a group  like  this,  I believe.  Is  to  achieve  a procurement  irmthodology  or 
I a set  of  procurement  measures  that  meet  a number  of  crIteHa.  They  must  be  creditable 
with  the  Industrial  sources,  flexible  to  meet  current  arid  future  Department  of  Defense 
requirements,  capable  of  being  understood  and  being  Implemented  at  the  lowest  level 
procurement  activity,  and  capable  of  opening  the  coemxjnl cation  links  and  keeping  them 
( I open  between  the  user  activity  and  the  procurer.  I would  add  to  that,  that  they  should 
be  economical  to  implement  and  administer  and  they  should  be  within  the  Intent  of  the 
statutes  that  govern  the  expenditure  of  taxpayers  money.  A goal  for  this  symposium  could 
^ then  be  greater  ecorxxqr  through  better  procurement  methods. 

Again  a welcome  to  West  Point.  I hope  your  stay  will  be  both  Informative  and  rewarding. 

Ue  wish  you  well  In  the  Important  task  that  you're  undertaking.  Thank  you  very  mKh. 
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Director  for  Procureaent  and  Production 
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Oood  Homing  ladles  and  gentleaen. 


On  behalf  of  the  An^r  Naterlel  Oevelopaent  and  Readiness  Coamnd  and  Its  CoMondlng  General, 
General  Jack  Guthrie,  I aa  pleased  to  welcoae  you  to  this  6th  Annual  DOO  Procureaent 
Research  Syaposlui  and  I feel  strongly  that  the  discussions  that  will  ensue  here  will 
enhance  the  "Translation  of  Procureaent  Research  Into  Actloa,”  the  theae  of  this  year's 
syigMsIiai.  As  we  learn  about  coapleted  and  on-going  research  projects  and  through 
discussions  of  vital  relationships  between  the  procureaent  research  and  operational 
spheres,  we  should  be  able  to  generate  Ideas  and  st1«ilate  actions  which  will  visibly 
laprove  the  efficiency  of  the  Govemaent's  procur«4Kat  systca. 


* * 

1 aa  pleased  that  this  year's  syaposlua  Is  not  restricted  to  procureaent  and  acquisition 


research  conducted  by  the  allltary  services,  but  that  other  Governaental  agencies. 
Industrial  concerns,  and  acadealc  Interests  are  also  well  represented.  Thus,  this  year's 
syaposlua  will  not  only  provide  for  a greater  breadth  of  technological  transfer  of 
laproved  procureaent  methods  and  aanageaent  techniques,  but  It  will  set  the  sUge  for 
wider  based  procureaent  research  symposia  In  years  to  coae.  Future  conferences  In  this 
series  will  be  conducted  on  a broader  governaental  basis,  probably  by  the  Federal  Procure- 
aent Institute,  and  have  an  even  wider  participation  than  In  this  and  past  years. 

I have  a double  pleasure  being  with  you  today.  The  first  Is  returning  to  West  Pofnt  where 
I spent  8 of  ay  first  14  years  In  the  Angf,  4 as  a cadet,  and  4 as  an  IriStructor  In  the 
Department  of  Mechanics.  The  second  Is  a result  of  ey  present  Job.  I have  under  wy 
operational  control  a research  organization  to  pursue  the  developaent  of  new  procurement 
concepts  as  well  as  to  assist  me  In  the  performance  of  wy  procureaent  operational  and 
management  duties.  The  pleasure  of  the  f1r>t  will  be  over  In  a few  days,  except  of  course 
for  the  renewed  nenorles  and  stlaulatlng  experlerxes.  But  the  pleasure  and  challenge  of 
the  second— I am  just  beginning  to  experience.  How  I view  my  role  In  this  new-found 
responsibility  Is  what  To  going  to  relate  to  you  this  morning. 

When  I was  selected  as  Director  for  Procurement  and  Production  for  the  Materiel  Development 
and  Readiness  Command,  one  of  my  first  actions  was  to  review  organization  and  mission. 

In  addition  to  the  duties  and  responsibilities  assigned  to  me  for  directing  the  procure- 
ment and  production  activities  of  the  Materiel  Development  and  Readiness  Command,  I found 
that  I was  to  exercise  operational  control  over  the  Anqy  Procurement  Research  Office.  I 
contemplated  what  my  responsibilities  would  be  for  exercising  this  control  and  how  It 
might  relate  to  ngr  other  responsibilities  as  director  of  procurement  and  production  activi- 
ties. From  my  previous  experience  as  a procurement  operator  and  manager  and  my  academic 
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exp>sure  as  a faculty  setter,  I recognize  the  need  for  balance  betueen  operational 
relevance  and  scientific  authenticity  In  the  conduct  of  procuraviit  research.  I realize 
^ tuo  are  not  Incoepatible,  but  the  pressures  of  tlee  sonetlaes  wLcs  the  reconc111a> 
tloe  vei^  difficult.  This  year  we  are  hawing  In  on  "translating  procurewent  research  Into 
**  * *2  explain  hew  I conceived  of  the  relationship  between  ^y  responsl- 

««  M?*  **  * Director  of  Procureaent  Policy  and  Operations  and  py  responsibility  to  •eke 
effective  use  of  our  procurewnt  research  resource. 

I****?*^???**"^"*  waMgewnt  with  research  would  nomally  require  a definition  of  each. 
But,  1 teller  the  Managerial  understanding  of  this  assenbly  will  enable  m to  forego  a 
discussion  of  procureaent  aanageaent.  1 will,  however,  offer  a bHef  definition  of 

" as  to  provide  you  with  py  perspective  of  the  syaposlua's  subject. 

I shall  offer  a short  definition  of  translate  and  action  so  that  I can  ctMolete  the 
linkage  explicit  In  our  theae  this  year. 

Procurewnt  research  Is  the  Identification,  analysis  and  developaent  of  approaches  to  the 
4 of  probleiB  faced  by  acquisition  planners,  procureaent  nanagers  and  contracting 
officials.  The  actual  research  process  proceeds  froa  problea  confireatlon  through  research 
collecttoa*  analysisp  and  Interpretation  to  developaent  of  laplenentatlon  or 
^rational  testing  procedures.  It  Is  the  latter  function,  Inplewntatlon,  where  trans- 
lating coaes  In. 


The  Uebster  International  Dictionary  definition  of  translate  Is  on  the  viewgraph.  It  Is 
laportant  to  know  that  In  the  context  of  this  synposlua  tie  aean  all  of  these.  It  Is  nec- 
essanr  to  translate  the  scientific,  rigorous  language  of  a research  report  Into  aore 
com  opmtional  torus  for  understanding  and  use  by  personnel  working  In  a less  stnic- 
tored  envirmnt.  The  research  concepts,  objectives,  and  Inferences  aust  be  converted  to 
toras  coapatible  with  the  work  situation  our  procureaent  operators  find  thonselves  In. 

The  iaproved  or  new  techniques  ajst  be  moved  froa  the  laboratory  to  the  acquisition  plamlng 
srene  or  procureaent  office.  Lastly,  action  Is  sinply  "the  process  of  doing;  an  exertion 

tETTESnsIsts  of  developing  strategies  and  plans, 
dieting  and  controlling  procureaent  actions,  and  aanaglng  the  resources  needed  to 
perform  these  actions. 


o 


OEFINITIONS 

■Procur«ient  Research"  Identification.  Analysis,  and  Oevelopoent 

of  approaches  to  problems  faced  by  acquisition 
planners,  procurement  managers,  and  contracting 
officials 


"Translate" 


"Action" 


Turn  from  one  language  to  another,  change  to 
another  form,  move  from  one  place  to  another 
(Webster's  International).  We  mean  all  three. 

Process  of  doing,  exertion  of  pomer  or  force 
I.e..  developing  plans,  directing  and  executing 
procurement  actions g managing  resources 


o 

('t 


planners,  researchers,  aaiugers  and  operators 

to  collaborate  In  this  translation  process. 

He  might  liken  the  transition  process  to  the  "marketing"  function.  If  we  coavare  the 
Mrket  to  uie  procureaent  or  acquisition  system  and  customers  to  managers  and  operators 
Whose  needs  Include  new-fouiri  knowledge  and  products  to  satisfy  the  system's  needs,  then  we 


set  the  vital  interdepen^e  of  procurewnt  research  and  the  aarket.  To  varying  degrees 
tie  stare  the  responsibility  for  the  aiarketing  of  procure»nt  research.  I don't  nean 
Mlling,  but  deteraining  the  needs  of  the  procurenent  systca.  identifying  and  directing 
the  resources  required  to  aeet  these  needs,  and  inploanting  the  results. 


TRAHSUTIOh*  AS  A 'ItMUnilK*  nflCTIOl 


•MJaiT 

Acmnsmow  systoi 

Planners 

Managers 

Procurcaent  Officials 
Contractors 


TRAMSLATIOr 
MARgTIIIC  HAtOttMEIfT 

Need  Assessaent 
Priority  Estahlishaent 
Resource  Ninageaent 
laplcaentation  Planning 
Perforaance  Evaluation 


'PROCUREMENT  RESEARCH' 
SClEinFIC  INTESnSATION 

New  Knowledge 
New  Techniques 
New  Insight 


days,  you  irt  11  be  seeing  actual  products  that  have  gone  through  this 
^i^tl^  proems,  ranging  fro«  Jla  SUnsbenry's  apprehensive  effort  In  developing  and 
Introducing  a stlvilatlng  new  profit  approach  to  deUlled  technique  for  neasuHna  pro- 
^rewnt  woi^force  productivity  and  establishing  pricing  Indices.  In  between,  we  will 
witfiKS  a wide  amy  of  Innovative  procur«Knt  concepts  and  techniques,  all  extrwly 
In^rt^  to  the  »iage»rt  of  system  and  mterlel  acquisition.  For  the  first  tine  In 
a 5* session  on  Industrial  procuremnt  research,  giving  us  an 
^portunlty  to  coi^ re  Ideas  and  procedure  In  an  envlrorwient  of  challenge  and  cooperation. 
In  one  smsion.  we  re  looking  at  ourselves  to  see  where  we've  been,  where  we  are  and  where 
we  1^  going.  In  another,  we  exaalne  our  mst  important  comodity,  ourselves  — the  procure- 
■ent  range  as  well  as  the  specific  content  of  these  research  presenta- 

tions. I believe,  demnstrate  that  there  are  few  limits  to  areas  In  which  procurement 
research  can  assist  planning  and  management. 

^ ^ examples  and  suggestions  for  Improving  research  utilization  which 
wil  ^ presented  here,  r.d  like  to  suggest  a particular  opportunity  for  promoting  transla- 
tlon  Into  action  to  idilch  we  should  be  giving  greater  attention. 

WIthMt  getting  them  Immersed  In  firefighting,  we  need  to  Involve  procurement  research 
^ple  In  the  shaping  and  Implementation  of  our  management  goals.  This  Involvement  on 
their  part  can  serve  to  generate  frjitful  research  topics,  which  not  only  assures  relevance 
but  generates  organizational  support  as  well.  Theory  building  Is  also  aided  by  this 
association,  for  the  researcher  Is  permitted  to  work  on  problms  In  systm  context  as 
opposed  to  Isolated  procurement  Issues. 

From  the  other  side  of  the  fence  I believe  we  need  to  make  greater  use  of  our  battle-hard- 
ened procurement  managers  and  operators  In  procurement  research.  I know  that  some  of  you 
use  these  managers  and  operators  to  conduct  procurement  research  on  occasion,  but  a system- 
atic appmeh  will  yield  more  comprehensive  results.  If  planned  tours  of  duty  are  Impractl- 
wl,  assignment  to  limited  part-time  projects  still  provides  a valuable  stimulus  to  the 
tra^latlon  process.  People  who  have  helped  form  research  objectives,  designs,  and 
analyses  plans  are  In  a substantially  better  position  to  assess  performance  and  Implement 
action  than  those  idio  receive  results  of  research  cold. 
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, . ■ (rnKTUilTIES  FOR  MXENTIMTIN6  THE  TRANSLATION  PROCEks  ' ‘ 

Use  procurcKnt  reseirth  personnel  In  sheplng  and  lapleaentlng  or^nlzitlonal  goals: 

Generates  Fruitful  Topics 
Gains  Organizational  Support 
Aids  Theory  Building 

Use  manager  and  operators  In  procurement  research:  ^ 

Enhances  Performance  Imdicator 
Speeds  Implementation 

Insures  Advocacy  ^ 


The  opportunities  for  Improved  covainicatlons  and  undersUndIng  afforded  by  reciprocal  use 
of  researchers  and  manager/operators  la  each  ether*s  envlronmefit  far  outweigh  che  aMnl- 
strative  difficulties  associated  with  the  assignments.  Ue've  tried  It  on  a limited  basis, 
and  It  works.  This  series  of  symposia  has  generated  a tremendous  amount  of  cross-»funct1onal 
understanding  and  cooperative  arrangements;  but.  In  between  ametlngs  we  can  enchance  the 
research/action  ratio  even  more  by  Inducing  greater  cross-usage  of  our  personnel  within 
organizations.  

Having  underlined  the  essential  relationship  between  procurement  research  and  procurement 
action.  I feel  constrained  to  offer  a note  of  caution.  He  can  translate  all  we  want  and 
cooperate  to  the  utmost,  but  If  the  results  of  a procurement  research  project  do  not  meet 
the  system  needs,  we  must  recognize  this  for  what  It  Is.  The  real  problem  may  de^ 
current  solution,  the  research  may  be  Inadequate  or  the  Implementation  may  be  Improper. 

But.  whatever,  we  ajst  avoid  the  beartrap  of  expecting  that  all  research  will  lead  to 
action. 

1 know  all  of  you  know  or  have  heard  this  caveat  before,  but  with  our  theme  this  year  being 
on  results.  I think  the  message  bears  repeating. 

We  may  well  be  standing  on  the  threshold  of  a new  era  In  government  procurement.  A top 
level  executive  office  of  procurement  policy  has  been  esUbllshed.  a federal  Institute  for 
procurement  1$  being  set  up.  and  Congress  and  the  private  sector  are  challenging  govern- 
mental procurement  to  Improve  Its  procurement  practices  and  find  new  cost-effective  ways  of 
doing  business.  Procurement  research  may  be  the  caUlyst  for  us  to  meet  the  challenge  and 
fulfull  the  promise  of  the  new  organizational  arrangements. 

NEW  ERA? 

Top  Level  Executive  Policy  Office 
Federal  Procurement  Institute 
Congresslonal/Publlc  Challenges 
Procurement  Research  - The  Catalyst 
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The  next  fw  days  should  go  « long  way  In  noving  us  to  a better  understandlim  i-n.  . . ■ 
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B6  Stansberry's  paper  Is  not  available  for  publication:  His  presentation  was  based  on 
lessons  learned  frow  Pfofit  '76,  a najor  OOO  procureaent  research  project.  As  a result  of 
Profit  '76,  General  Stansberry  fonailated  sowe  *Keys  to  Success*  for  trifislatlng  procure- 
nent  research  efforts  Into  positive  action  and  presented  these  Insights  to  the  syii|>os1iai 
audience.  The  following  are,  very  briefly*  the  oaln  points  of  his  presentation. 


The  first  key  to  success  Is  to  work  on  real  pr^lens.  Resources  are  too  United  for 
the  procurenent  research  coMunlty  to  concentrate  Its  assets  on  ”non-prob1cns*  that  nay  be 
very  Interesting  but  nean  nothing  to  one's  particular  organliatlon. 


The  next  key  to  success.  General  Stansberry  believes.  Is  that  If  one  Is  ever  going  to  do 
substantial  research.  It  Is  Inportant  to  have  a charter  qlvinq  oneself  authority  ami  respon- 
( ) siblllty  which  Is  signed  by  someone  Inportant.  This  enables  the  researcher  to  avoid  the 

bottlen^  often  encountered  when  dealing  with  aoy  bureacratic  organization. 


Thirdly,  It  Is  necessary  to  find  sone  very  Ulented  people  and  generate  soig  enth^lasn. 
Having  the  best  people  do  research  will  eventually  ligwove  the  whole  process  of  how  D06 
"does  business." 


One  of  the  nost  Inportant  keys  Is  to  plan  well  before  starting  a job.  The  essence  of 
getting  ai^thlng  done  Is  to  lay  out  a good  plan  and  check  It  carefully  before  beginning  wort. 
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Another  key  to  success  - one  In  which  General  Stansberry  demands  when  he  Is  In  charge 
of  a project  - is  to  Involve  the  people  In  the  field  ^ ac^lly  do  the  job.  They  can 
offer  unique  and  valuable  Insights  Into  the  real  problems  affecting  their  jobs. 

Using  the  library  builds  up  an  authoritative  data  base  and  precludes  the  researcher 
from  doing  wort  that  has  already  been  done  and  Is  therefore,  a key  to  success. 

A seventh  key  to  success  for  the  procurement  researcher  Is  to  allow  those  idw  hold 
opposing  biases  to  exanine  and  attack  the  nuDt)ers  and  data  that  he  has  githered.  This 
seizes  as  an  objective  check  against  mistakes  that  may  have  been  owde  and  familiarizes  one 
with  the  "other  side.* 


Creating  a Climate  of  Acceptance  by  "steering*  the  policy  maker  toward  one's  conclusions 
Is  also  a significant  key  to  success.  A climate  of  acceptance  can  enable  the  researcher  to 
{ } make  others  realize  that  their  conclusions  are  not  necessarily  as  they  think. 


For  the  project  manager,  an  obvious  but  very  necessary  key  Is  to  become  heavily  inwlved 
in  the  project  so  as  to  kyy  and  un^i^tand  the  details.  The  manager  should  at  least  oe 
able  to  de^te  the  data  with  his  critics. 
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th^  one  Of  the  weakest  areas  of  procurewnt  research  Is  ' j 1 
research  wrk  causes  sone  chaoses  In  the  real  world.  It's  useless 

floUhSThll  rijortT***'^  ****  "***'**“'*  *®  ^rhet  his  product  once  he  ^ 'v 

riianS^  Sid>stantlal  Involves  a degree  of  Hsk.  In  order  to  ske^ 

!*•*  procurenent  research.  It  Is  InporUnt  to  take  a few  chances  ^ 

idilch  Is  General  SUnsberry's  last  k«y  to  succcs^  . ” — * * CMoces.  rr  ^ ^ 
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Or.  Richard  J.  Tralnor  > 

Director  of  Sjrstess  Review  and  Analysis 
Office  of  the  Deputy  Chief  of  Staff  for  i 

Research.  Oevelopoent  and  Acquisition.  US  An^ 

. ^ V :j 


s» 

- r 


Dr.  Tralnor  spoRe  extemporaneously  on  his  subject  and  did  not  submit  a paper  per  se.  The 
following  abstract  and  charts  should  give  an  Idea  of  his  talk: 


The  major  weapon  and  space  systems  developed  and  pn>d*jced  by  the  aerospace  Industry  are 
at  the  world's  technological  forefront.  The  aerospace  Industry  and  government  use  the  very 
best  Ulent  to  manage  the  acquisition  process.  In  spite  of  this,  the  major  system  acquisition 
process  continues  to  be  characterized  by  substantial  cost  growth  and  schedule  slippage. 

The  development  phase  of  the  acquisition  process  Is  becoming  Increasingly  Important  and  Is 
the  subject  of  this  study. 


The  study  of  the  causes  of  cost  growth  has  been  approached  from  several  aspects.  A 
review  of  the  literature  and  examination  of  current  management  practices  provided  one  focus. 
This  paper  will  cover  the  multiple  and  highly  Interrelated  causes  of  cost  growth.  It  will  * 
exMlne  the  cost  growth  of  nine  000  and  NASA  weapon  and  space  systems. 


And  lastly  the  paper  will  discuss  the  developaient  of  descriptive  and  predictive  models 
of  the  development  process.  The  descriptive  model  Is  oriented  toward  description  of  the 
Impact  of  a change  In  management  strategy  on  the  cost.  time,  and  risk  associated  with 
system  development.  The  predictive  model,  a derivative  of  the  descriptive  model,  attempts 
to  estimate  the  total  cost  of  developing  a system  based  on  data  available  early  In  the 
system's  development  cycle. 
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• DEFiryiTION(S) 

• RECEIPT  CASE  HISTORIES 

• PRII^CIPAL  CAUSES  OF  COST 

GROWTH 
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COST  GROWTH  DEFINmOIM(S) 

» «w  * 

r •• 

•p,  .*'■•’ 

VARIATION  IN  COST  WITH  RESPECT  TO: 
CONTRACT 

THE  SYSTEM  REPLACED 
CONTROLLABLE  COSTS 
EXCLUDING  OVERHEAD  ' ' ^ 

THE  SOOTH  SYSTEM 
THE  ESTIMATE  AT  ENTRY  INTO  ED 
THE  ESTIMATE  AT  PROGRAM  INITIATION 
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RECENT  CASE  HISTORIES 
(PRODUCTION  PHASE) 
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MAIN  CAUSES  OF  COST  GROWTH 
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• REQUIREMENTS  FORMULATION  PROCESS 

• TECHNICAL 

• PLANNING,  PROGRAMMING,  BUDGETING 

PROCESS 

• COST  ESTIMATING  TECHNIQUE  ' 

• MANAGEMENT  INCENTIVES/INACtIoN 

• PRODUCTION  TRANSITION 

• ESCALATION 
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MAJOR  RESEARCH  PROBLEMS 
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1.81 
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*fl4  months 

1.16 
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-actual  uovoiopnionc  Timo  t lalCUil  sttsmatt 
‘Avoraco  of  Two  Early  Estlmatei 
^Average  of  LANDSAT  1 and  2 

Ocvclojxnent  cost  as  of  December  31,  1976  was  1740.4  Million,  the  added  |4.9  Million  erowth  occurins 
ofter  fleet  deployment.  . • • • • • • . * 

EKcludes  $3?. 9 Million  of  development  funded  with  preeursmanc  funde,  end  eennleter  end  eepsule  program. 
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TABU  14 


tllMMARy  or  rAOTOXS  CAU8IN0  COST  GROWTH 
FOR  MINI  000  AND  NASA  SYSTEMS 
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SUMMARY 


LARGE  COST  GROWTH  CONTINUES 
EXTENT  OF  GROWTH  IS  DEPENDENT  ON  DEFINITION 
DETAILED  ANALYSIS  OF  CAUSE  • EFFECT  IS  IMPOSSIBLE 
CONSIDERABLE  RESEARCH  RESULTS  ARE  AVAILABLE 
MAIN  CAUSES  OF  COST  GROWTH: 

••  REQUIREMENTS 
•• PPBS 

••  TECHNICAL  PROBLEMS 
••  INCENTIVES 
••  ESTIMATING  TECHNIQUES 
COST  GROWTH  VERSUS  THE  ALTERNATIVE 
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* CCMPARATIVE  ilim.TSIS  OF  POO  iWD  WftSA  CONTIWCT  COST  OWCaWES 
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Itomiityd#  of  Pnocuf  nt  Decisions  in  the  Federal  Budget 

Projected  expenditures  of  the  Federal  Government  of  the  United  States  during  the 
five-year  fiscal  period  from  October  I,  1977  through  September  30,  I98I  total 
2,272  billions  of  dollars.'  Of  this  total,  purchases  by  the  U.  S.  Department  of 
Defense  average  $63*3  billion  per  year  and  those  by  the  National  Aeronautics  and 
Space  Administration  average  $2.5  billion  per  year for  a cond>ined  total  of 
$66.8  billion,  or  about  15%  of  the  projected  annual  Federal  outlay. 

In  terms  of  Aerospace  Vehicles  alone,  U.  S.  Government  orders  amounted  to  $18.6 
billion  for  1975*  representing  21%  of  the  total  Department  of  Defense  and  National 
Aeronautics  and  Space  Administration  outlays  of  $88.6  billion  for  that  year. 3 
Contract  awards  exceeding  $10,000  each  for  Fiscal  Year  1975  totaled  $36.8  billion 
for  the  Departoen:  of  Defense  and  $2.0  billion  for  the  National  Aeronautics  and 
Space  Administration.  Of  these  totals  $6.3  billion  In  Defense^  and  $1.3  billion 
in  NASA^  were  negotiated  Incentive  type  contracts  with  private  industry,  and 
$3*8  billion  in  Defense  and  $0.3  billion  in  NASA  were  negotiated  fixed  fee  con- 
tracts with  private  industry.  Thus,  of  the  combined  total  contracts  of  $88.6 
billion,  $5.6  billion,  or  about  65,  were  incentive  type  contracts  and  $A.I  billion 
or  about  55,  were  cost  plu*,  fixed  fee  contracts. 


Given  that  approximately  11%  of  the  DOO  and  NASA  contracts  were  of  the  incentive 
or  fixed  fee  type,  and  that  these  contracts  represent  about  15%  of  U.  S.  Federal 
Government  expe^icures,  a simple  extropolation  would  indicate  that  about  1.65% 


Executive  Office  of  the  President,  The  Budget  of  the  United  States  Government. 
Fiscal  Year  1977.  Part  3»  Economic  Assumptions  and  Long  Range  Budget  Projcc- 
tions,  pp.  35*36.  (Washington,  0.  C.:  U.  S.  Government  Printing  Office,  1976.) 

^Ibid. 

^United  States  Department  of  Commerce,  Bureau  of  Economic  Analysis,  Survey  of 
Current  Business.  Volume  56,  No.  0,  pp.  S-19  and  S-AO.  (Washington,  0.  C.: 

U.  S.  Government  Printing  Office,  August  1976.) 

Office  of  the  Secretary  of  Defense,  Military  Prime  Contract  Awards  Fiscal  Year 
J975,  Table  16.  (Washington,  0.  cT.  OASD  Comptroller,  15  October  1975.) 

^National  Aeronautics  and  Space  Administration  Headquarters,  Annual  Procurement 
Report,  Fiscal  Year  197^.  p-  17-  (Washington.  0.  C.:  NASA  Assistant  Admin- 
istrator for  Procurement,  I97l».) 


37 


of  projected  Federal  expenditures  of  S2.272  billion  are  candidates  for  these 
types  of  contracts.  This  represents  a forecasted  volume  as  high  as  $37.5  billion 
over  the  next  five  years  for  the  Department  of  Defense  and  MASA  alone.  Outlays 
of  other  Federal  agencies  In  similar  procurements  would  have  the  effect  of  in- 
creasing this  total. 


Given  the  projected  magnitude  of  government  procurements  which  are  susceptible 
to  these  cost  reimbursemei.t  type  contracts,  research  and  analysis  directed 
toward  improvements  In  the  mathematics  and  micro-economics  of  the  relationships 
between  estimated  and  actual  costs  in  these  contracts  can  have  significant  Im- 
pacts on  the  efficacy  of  governmental  purchasing  activities  in  the  future.  This 
paper  Is  an  empirical  comparative  analysis  of  such  cost  outcomes  based  on  a 
random  sample  of  174  NASA  and  300  Army  contracts  completed  during  the  period 
1964  to  1975.  . 


The  Market  Mechanism 


•*  t 


As  noted  by  Peck  and  Scherer^  In  1962,  reiterated  by  the  Commission  of  Government 
Procurement'  in  1972  and  by  Fox#  in  1974,  the  market  systcia  in  which  the  govern- 
ment operates  differs  from  the  private  industrial  market  In  several  crucial  ways. 


First,  the-mark^Js^ determined  weakly.  If  at  all,  by  supply  and  demand.  Rather. 
Congress  determines  annually  Sow  much  will  be  spent  by  the  government  on  research, 
development  and  production  of  .'<ajor  systems.  These  determinations  are  based  not 
so  much  on  what  may  be  offered  in  the  narket  place,  but,  rather,  by  political  and 
economic  conditions,  international  events  and  the  interests  of  the  Congressmen 
and  Senators  who  serve  on  defense,  space,  budget,  appropriation  and  other  conrnit- 
tees  and  subconni ttees. 


Secondly,  the  prices  of  these  procurements  are  not  usually  detemined  by  normal 
market  competition.  Cost* reimbursement  and  incentive  contract  prices  depend 
ultimately  on  costs  actually  experienced.  The  contractor's  profit  is  initially 
determined  by  the  level  of  costs  estimated  when  the  contract  is  negotiated,  and 
is  subsequently  renegotiated  each  time  a contract  change  affecting  price  occurs. 
Such  changes  occur  frequently,  especially  in  the  typically  large  incentive  and 
cost- reimbursement  contracts  calling  for  advances  In  the  technological  state  of 
the  art. 


Thirdly,  because  the  government  is  usually  the  only  buyer  and  investment  barriers 
to  entry  limit  the  number  of  suppliers,  the  market  is  frequently  referred  to  as  a 
"bilateral  monopoly"  in  that  the  buyer  and  seller  exist  interdependent ly;  neither 
can  achieve  its  principal  objectives  without  the  other. 


Peck.  Merton  J.  and  Scherer.  Frederic  M, . The  Weapons  Acquisition  Process:  An 
Economic  Analysis.  (Cambridge.  Hass.:  Harvard  University  Press.  1962.) 

^McGuire.  E.  Perkins.  Chairman.  Report  of  the  Commission  on  Government  Procurement. 
(Washington.  D.  C.:  U.  S.  Government  Printing  Office.  1972.) 

8Fox.  J.  Ronald.  Arming  America.  Hw  the  AJ.  S.  Buys  Weapons.  (Boston.  Mass.: 
Harvard  University  Press,  197I*  ) • 
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TABLE  I.l.  SUMMMOr  OF  DlFFERFIKr  S IN  CIIAIL\crF.RISTlCS  BETVEES  THE 
PRIVATE  AND  PUBLIC  INDUSTRIAL  MARKET 

Market’  Characteristics  ‘ ^ 


Private  Market 

The  aclier  initiates  new  product 
innovations  based  on  analyses  of 
potential  markets.  He  has  no  certain 
knowledge  of  a product*8  saleability. 

The  buyer  has  a vide  range  of  choice 
between  products  in  the  soae  category 
that  have  real  or  advertised  differ^ 
ences. 


Price  is  a doninant  factor  in  a 
buyer's  choice  because  adequate 
substitute's  for  a product  are  often 
available. 

The  market  tends  to  be  impersonal. 
Bayers  and  sellers  act  independently. 

The  producer  finances  the  develop- 
ment-production 'effort. 


The  market  usually  contains  several » , 
or  many,  customers  for  each  prod*.ict. 

Prices  arc  primarily  determined  by 
competition. 

Demand  is  either  relatively  constant 
(e.g.»  for  staples) » or  tends  to  be  a 
function  of  disposable  income  (e.g., 
for  non-essentials). 

The  basic  design  of  the  product 
changes  slowly  and  requirements 
for  a given  model  are  relatively 
stable. 


Public  Market  • 

‘ * . 

The  buyer  establishes  the 
requirements  for  a product.  The 
producer  then  begins  development 
and  prodmetion.  • « . 

...  ; — .. 

Eelatively  few  products  are 
produced  simultaneously  for  the 
same  mission.  Although  Che  buyer 
sometimes  has  the  option  to  choose 
among  prototypes » the  time  and  cost 
of  producing  new  systems  once  pro- 
duction has  begun  discourages 
replacement. 

Price  is  only  one  of  the  factors 
that  govern  a customer's  choice. 

It  may  be  far  less  important  than 
qualityp  availability^  or  tccfaeology. 

^ The  market  is  highly  personal.  The 
buyer  has  constant  contact  with  the 
seller's  organization. 

The  buyer  bears  most  of  the  develop- 
ment cost  and  may  provide  equipment 
and  facilities  for  the  use  of  the 
producer. 

The  market  is  esscmtiaily  one- 
customer  (monopsonlstic). 

Price  is  determined  by  an  evaluation 
of  anticipated  and  actual  costs. 

Demand  is  a function  of  the  tech- 
nology available,  or  of  c^stlmatcs  of 
a potential  enemy's  technological 
resources. 

The  product  nay  be  technologically 
obsolete  before  production  is 
completed. 


Adapted  from  CleJ.ind.  David  I.  and  King,  William  R.,  •’The  Defense  Market 
System,"  Defense  Industry  Bulletin.  January  1968. 
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TABLE  1-2.  KPA.lTIJ£flT  OF  DEFOiSE 
AWARDS  CY  COJO’ETITIOII  & STATUTORY  AUTHORITY, 

. FISCAL  YEAR  1976 

(Suxarized  froa  "Military  Prime  Contract  Awards  Fiscal  Year  1976", 
Office  of  the  Secretary  of  Defense  (Comptroller),  Washington  D.C.) 

.3  amount' 

(Billions) 


PERCENT 


TOTAL 


.Coopetitive  Subtotal 

Formally  Advertised 
Negotiated 

Non-Competitive  Subtotal 

Follow-on  After  Competition 
Sole-Source 

Negotiation  Authority:  (14.2  * 23.4 


$37.6) 


(1) 
|! 
4) 
[5 

(8 
(9 
r (10 

12^ 

13 

14 

15) 

16) 
17) 


National  Emsrgenqr 
Public  Exigency 
Not  exceeding  $10,000  each 
Personal/Prof.  Services 
Educational  Institutions  ■ t 

Outside. of  U.S. 

Medical  Supplies  & Medicines 
Supplies  for  Resale 
Subsistence  Items 
Competition  Impractical 
Experimental,  Research,  Development 
Classified  Purchases 
Standardization  of  Parts 
High  Initial  Investment 
Negotiated  after  F.A. 

Mobilization  Base 
Other  Authorized 


40.8 

lOO.OX 

17.4 

42.6. 

3.2 

' 7.9" 

14.2-  ' 

‘ji’34.7 

23.4  ^ 

57.4 

5.8 

' 14.3 

17.6  w V 

v «.l. 

1 » (92. IX 

» 34.7  + 

. t 

2.2 

5.9 

2.2 

..  • 5.9 

2.5 

: G,6 

0.1 

' 0.3 

0.5 

. 1.3 

. 2.0 

:■  5.3 

0.1 

■ 0.3 

0.5 

1.3 

1.7 

4.5 

13.2 

35.1 

5.8 

15.4 

0.1 

0.3 

0.1 

0.3 

4.4 

• 11.7 

0.0 

0.0 

1.9 

• - 5.0 

0.3 

0.8 

r r-fj 


57.4) 


AWARDS  BY  l-iAJOa  "HARD  GOODS" 

- FY76  (Work  in 

U.S.) 

AF’OUiiT 

PERCENT 

Aircraft 

$ 7.0 

23.8 

Missiles  & Space 

4.7 

IC.O 

Electronic,  Communication 

4.5 

1S.3 

Ships 

3.8 

‘ 12.9 

Services  (to  Hard  Goods) 

3.2 

10.9 

Constructicn 

2.2 

7.5 

Hi  sc.  Hard  Goods 

1.3 

4.4 

Tank,  Automotive 

1.3 

4.4 

TUnmunition 

0.9 

3.1 

Weapons 

0.5 

1.7 

TOTAL 

$29.4 
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TABLE  1-3.  WATIOHAL  AERONAUTICS  AJID  SPACE  ADMINISTRATION 
• ■ ' ■ AHARDS  BY  CtttPETION  AND  STATUTORY  AUTHORITY, 


FISCAL  YEAR  1976 


^ -r 


u Troo  "NASA  Annual  Procurement  Program  Fiscal 

. Headquarters^  (Code  HA)  Washington,  C.  C.  205 


NASA  Headquarters^  (Code  HA)  Washington 


Year 
20546) 


i:U.i 
■ ■)?:; 

Mr 


>■'■  ■ • .,.'  ' •' AMOUNT  PERCENT  * 'O' 

•.  ;v;,  :w.  (Billions)  ..  . ’ 

llX  "^'l0p.0t  ( ) 


J •-  i • — 

• <'”*  i J ‘ T 

1'-  C«Bpetitive  Subtotal 

Formally  Advertised 
' 4/:  Negotiated 


viV.;  ;-:,  .?■  '•■  Non-Competitive  Subtotal  .>0.7  ‘ ‘ / ‘11' 

- .?•  • '•  ' ^ V '^  • ■■‘-  • ..  : ^ 4-h  Vo-' - 

^ v . ^ Negotiation  Authority:  (1.5  + 0.7  - $2.2)  - ( 85S  - 58  + 27) 

I? '’••■  r ■/ ‘i."  * fe,  L..  — 


‘ t. 


1 (1)  National  Emergency 
' (2)  Public  Exigency 
. .3)  Rot  exceeding  $10,000  each 

i[  5^)  Personal/Prof.  Services 
v'lS)  Educational  Institutions 
V‘’(6)  Outside  of  U.S. 

‘(7)  Medical  Supplies  & Medicines 
y (9)  Subsistence  Items 
; (10)  Competition  Impractical 
(11)  Experimental.  Research,  Development 
. (17)  Other  Authorized 


I . * 

- ^ 0.04 


..i'.,  Uj 

0.04  '»  ( 1.8 


^ * Less  than  0.5S 


Space  Flight 
Space  Science 
Applied  110.15 
Aeronautics 

Tracking  & Data  Acquisition 
Energy  Programs 
Non-Program  (services) 


AHARDS  BY  PROGRAM  OFFICE  - FY76  (Work  in  U.S.) 


TOTAL 
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The  Economics  of  Coverryneot  Contracting 


The  first  comprehensive  treatoent  of  economic  incentives  to  be  reviewed  Is  that 
by  Peck  and  Scherer. This  19^2  volume  presents  an  overall  economic  analysis 
of  the  %«eapons  acquisition  process  in  the  Department  of  Defense,  drawing  on  a 
series  of  case  studies  involving  seven  advanced  comoiercial  products  used  for 
comparison  with  twelve  advanced  lieapons  * aircraft  and  missiles  * covering  the 
period  1920  through  1959*  A principal  conclusion,  based  on  regressions  of  expert 
opinion  on  empirical  data,  is  that  "reasonably  consistent  ordinal  measurement  of 
« overall  weapon  system  development  program  performance  by  knowledgeable  persons  is 
feasible.'*  Although  not  necessarily  startling,  this  conclusion  tends  to  support 
the  thesis  that  empirical  or  objective,  as  opposed  to  subjective,  research  Is 
feasible  and  neither  counter-intuitive  nor  ant i -Judgmental • The  comprehensi ve 
work  by  Peck  and  Scherer,  among  other  reconiaendat ions,  made  a very  specific  pro- 
posal concerning  the  data  base  for  procurement  research:  they  strongly  recom- 
mended the  development  of  a %#ell  structured  data  gathering  and  analysis  organi- 
zation at  the  Secretary  of  Defense  level  to  provide  the  basis  for  improved 
contracting  decisions.  ^ ^ \ 


^Umi  St^tttloal  Distribution  of  tim  of  to  OtlwtW  CwW 


• . T 

In  1967  Moore. in  writing  specifically  on  incentive  contracting,  drew  a number 
of  Inferences  concerning  cost  growth,  the  remarkable  aspect  of  his  analysis  for 
the  purpose  of  this  research  is  his  treatment  of  the  sample  distribution  of 
variation  In  actual  costs  as  compared  to  estimated  costs.  His  data  are  presented 
as  follows:  ^ 

Cost  Overrun  or  Underrun  (X)  Number  of  Contracts  Percent 
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*®Peck,  Horton  J.  and  Scherer,  Frederic  H. , The  Weapon*  Acquisition  Profw: 

An  Economic  Analysis.  (Cambridge.  Hass.:  Harvard  University  Press.  T5?2.) 
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Although  Moore  assumed  normality  for  tr.is  distribution,  it  is  quite  clearly 
skewed  to  the  overrun  values.  The  mean  is  -3-5^  whereas  the  mode  is  -JZ,  using 
the  midpoints  of  the  closed  class  Intervals  and  ♦ 2SZ  for  the  open  end  intervals. 


The  most  recent  comprehensive  treatment  of  government  contract  cost  outcomes  to 
be  reviewed  is  that  by  Commins'2  published  in  1973-  His  data  base  consisted  of 
IIB  Army  contracts  completed  between  1965  and  1970  which  parallels  the  Army  data 
base  used  In  this  analysis.  All  of  his  contracts  had  a final  cost  of  at  least 
one  million  dollars.  His  bilateral  bargaining  loodel  was  essentially  an  exten* 
Sion  of  a doctoral  dissertation  by  BerholdO  with  relaxation  of  the  assumption 
that  the  contract  would  be  finalized  at  the  minimum  acceptance  level  for  the 
contractor.  Rather,  the  aspect  of  relative  bargaining  strengths  of  the  parties 
Is  introduced  as  an  exogenous  variable,  and  the  cost  outcomes  are  treated  as 
endoge^s  variables.  He  tests  his  model  in  both  the  fixed  price  incentive  and 
cost  plus  incentive  fee  contract  forms.  His  assumptions  include  the  propositions 
that  (IJ  defense  go^s  are  "normal"  goods  in  the  profit  and  organizational  view 
of  the  contractor,  (2)  the  expected  cost  to  the  government  of  a given  contract 
Is  a function  of  relative  bargaining  strengths  of  the  parties,  (3)  both  parties 
have  perfect  knowledge  of  each  other's  utility  functions,  and  (4)  all  decisions 
are  made  at  the  time  the  contract  is  initially  negotiated.  The  ef^irical  results 
support  the  proposition  that  cost  overruns  are  primarily  influenced  by  the  con- 
tractor s relative  bargaining  strength.  His  final  conclusions  all  relate  to 
extracontractual  incentives;  that  cost  overruns  are  a function  of  the  contrac- 
tor s degree  of  risk  aversion,  his  extent  of  'teral  hazard"  and  his  relative 
bargaining  strength.  As  in  all  the  literature  reviewed,  the  subject  of  non- 
normality of  the  distribution  of  contract  outcomes  is  not  addressed. 
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Further,  statistical  analyses  are  perf  rmcd  on  a sample  of  17^  NASA  and  300  Army 
contracts  to  compare  contract  type  and  cost  outcomes  as  related  tD  a nunber  of 
procurement  situational  variables*  In  addition  to  standard  descriptive  statis* 
^•tuational  variables  such  as  procurement  type,  commodity,  contractor  and 
amount  a^  date  of  contract  are  regressed  on  contract  type  for  the  entire  san^le. 
In  addition,  the  sample  is  stratified  by  contract  type  and  regressions  arc  per- 
formed on  the  ratio  of  final  to  negotiated  cost  as  a function  of  number  of 

modifications*  procurement  authority,  negotiation  authority,  commodity, 
and  amount  of  initial  contract.  Correlation  matrices  among  these  variables  also 
are  reported* 

" ' ‘ ehPIRICAL  RESULTS 

Description  of  the  Data  t • • 

The  empirical  data  for  the  Army  contracts  Included  in  this  study  %#cre  provided 
by  the  U.  S.  Army  Procurement  Research  Office*  The  data  are  dravm  froo  a random 
sample  of  300  contracts,  each  In  excess  of  $500,000,  completed  between  1964  and 
1971*  Collect!^  of  this  data  base  required  15  man  months  of  effort,  and  was 
used  as  the  basis  for  a report  on  contract  cost  growth  in  major  Army  programs. 

The  findings  and  recommendations  of  the  report  included  the  observations  that 
of  additional  work  amounted  to  58^  of  the  total  sample  cost  growth, 
engineering  changes  almost  always  increased  the  contract  price,  and  that 
although  under-runs  occurred  more  frequently  than  over-runs,  the  total  Mount 
of  over-runs  far  exceeded  that  for  under- runs* 
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The  total  final  cost  of  all  300  Army  contracts  represented  in  the  sample  is  in 
excess  of  two  billion  dollars.  The  total  includes  112  Firm  Fixed  Price  (FFP), 

53  Fixed  Price  Incentive  (FPI),  50  Cost  Plus  Incentive  Fee  (CPIF),  82  Cost  P»us 
Fixed  Fee  (CPFF),  and  3 Cost  Plus  Award  Fee  Contracts. 

The  empirical  data  for  the  NASA  contracts  included  in  the  total  sample  were 
obtained  from  two  sources.  A sample  of  150  contracts  was  obtained  from  the 
Ames  Research  Center  (ARC);  the  balance  of  2k  large  cost  reiabursable  contracts 
MS  obtained  from  NASA  Headquarters.  The  ARC  sampK*  (.«.isists  of  85  fixed  price 
and  65  cost  reimbursement  type  contracts.  All  2li  of  tie  NASA  Headquarters  sample 
were  of  the  cost  reimbursement  type,  including  6 CPFF,  CPIF,  and  13  CPAF  con- 
tracts.  Unlike  the  Army  data,  the  NASA  data  base  is  not  complete  with  respect 
to  contract  type,  contractor  identification,  commodity,  negotiation  authority, 
nor  procurement  type  (Research,  Development,  Production,  Services,  or  Study). 

As  a result  the  regression  analyses  are  more  limited  for  the  NASA  contracts. 

In  addition,  the  aggregate  analyses  by  contract  type  are  limited  for  the  NASA 
contracts  to  fixed~price  (FP)  or  cost  reimbursement  (CR)  categories  whereas 
separate  analyses  were  possible  for  the  Army  contracts  for  FFP,  FPI,  CPIF  and 
CPFF. 
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DESCRIPTIVE  STATISTICS 


through  1 1 1-6  summarize  the  results  of  the  standard  descriptive 
statist  cs  ca  culated  from  the  data  available,  (llote  that  these  tables  and 
all  analytical  tables  to  follow  are  included  in  a technical  aopendix.  This 
appendix  is  not  included  in  this  paper  because  of  space  limitations.  Inter- 
ested readers  may  write  to  the  authors  for  a copy  of  the  technical  appendix.) 


deviations,  and  measures  of  skewness  and  kurtosls  are  reported 
for  all  contracts  in  the  sample,  as  well  as  on  a comparative  basis  bV 
Agency  (Ar^  ^SA)  and  by  contract  category  (FP,  CR).  In  addition.  Tables 
III-7  and  II 1-8  sumnarize  the  data  for  Army  contracts  only,  by  contract  type. 


Table  lll-l  displays  the  basic  descriptive  statistics  for  the  total  sample  and 
as  stratified  by  agency  and  contract  category  for  the  initially  negotiated 
^tract  amounts.  It  is  noted  that  the  arithmetic  mean  or  average  value  for  all 
53.0  million  with  a standard  deviation  of  $8.7  million  for  a 
coefficient  of  deviation  of  2.9.  The  Zk  HASA  contracts  have  by  far  the  highest 
mean  initially  negotiated  values  with  the  iSO  NASA  contracts  at  the  extreme  low 
end  of  the  mean  values.  An  unexpected  result  may  be  that  the  fixed  price  cate- 
gory  has  a higher  ne^  value  than  the  cost  reimbursement  category.  As  In  all 
dollar  cost  statistics  as  opposed  co  cost  ratio  statistics^  all  distribu- 
tions are  asymmetrical  and  skewed  to  the  higher  values.  The  kurtosis  values 
indicate  that  all  categories  are  leptokurtic  or  highly  peaked  at  central  or 
modal  amounts^  all  being  considerably  in  excess  of  3. 


Table  1 1 1-2  indicates  that  the  targe  NASA  Headquarters  contracts  resulted  in 
very  high  dollar  values  of  modifications  subsequent  to  initial  negotiation: 
a mean  of  $55.8  million  in  modifications  related  to  a mean  of  $57.5  million 
in  initial  negotiation  for  a 97%  modification  increase^  whereas  the  combined 
Army  sample  exhibited  a or  a 51%  average  increase.  Again,  all  values 

are  highly  peaked  and  skewed  to  the  right. 


Table  1 1 1-3  displays  the  adjusted  initial  amounts;  that  is,  the  initial  contract 
amounts  plus  the  subsequent  contract  modification  amounts.  Table  1 11-4  then 
shows  the  final  contract  amounts,  including  overruns/underruns  and  modification 
amounts.  The  mean  for  all  474  contracts  has  grown  from  $3.0  million  initially 
to  $4.6  million  for  a 53%  total  average  cost  growth.  The  Army  total  average 
cost  growth  is  53%  compared  to  111%  for  the  total  NASA  cost  growth. 


It  may  be  more  instructive,  or  at  least  easier  to  evaluate,  if  we  examine  ratios 
rather  than  absolute  amounts.  Tables  1 1 1-5  and  1 1 1-6  display  these  results, 
both  uncorrected  for  modifications  occurring  subsequent  to  the  initial  contract 
and  after  correction  for  the  dollar  value  of  modifications,  respectively.  The 
highest  ratio  cost  growth  again  occurs  in  the  24  large  NASA  contracts,  both 
corrected  (17.2%)  and  uncorrected  (140.9%).  The  Icx^est  growth,  as  expected,  is 
in  the  Hxed  price  category  showing  a 47%  uncorrected  growth  and  none  after 
The  NASA-ARC  contracts  show  a slight  reduction  In  cost  ratio  of 
•0-7%  after  correction.  The  growth  in  the  cost- reimbursement  contracts  is 
101.9%  uncorrected,  but  only  2%  after  adjustment  for  modifications.  The  Army- 
NASA  comparisons  show  nearly  equal  cost  growth  ratios:  Army  72.5%  uncorfected 
to  0.5%  corrected  and  NA^  73.3%  uncorrected  to  1.7%  corrected.  With  respect  to 
symmetry  of  the  cost  ratio  distributions,  all  but  the  fixed  price  category  are 
skewed  to  the  higher  values,  indicating  that  the  mean  or  expected  ratio  is 
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rlghcr  ihan  the  .tiodal  or  most  likely  value.  All  ratio  values  are  peaked,  with 
kurtosis  values  in  exccs;^  of  3*  In  measures  of  dispersion,  the  liASA  contracts 
show  higher  standard  deviations  than  the  Army  contracts. 

Referring  to  the  Army  comparative  analyses  by  contract  type.  Tables  III-?  and 
1 1 1-8,  it  is  noted  that  the  53  PPI  contracts  have  the  largest  average  value  and 
variation  In  amounts  initially  negotiated,  but  that  the  50  CPIF  contracts  domi- 
nate all  other  contract  types  once  contract  modifications  are  taken  into  account. 
With  reference  to  symmetry  of  the  distributions,  all  values  are  skewed  to  the 
right,  with  the  CPIF  contracts  In  every  case  having  the  largest  shift  of  the 
mean  or  average  values  to  the  right  of  the  !iodal  or  most  likely  (negotiated) 
value*  As  to  peakedness,  all  kurtosis  valces  exceed  3 Indicating  leptokurtic 
or  highly  peaked  distributions.  Again,  the  CPIF  contracts  dominate  in  this 
c<*aracterlstic. 

In  Table  1 1 1-8,  in  which  the  ratios  rather  than  absolute  amounts  of  overrun  and 
underrun  are  displayed,  it  is  noted  that  although  the  non- incentive  contract 
types  have  the  largest  mean  percentage  cost  overruns  before  adjustment  for  con- 
tract modifications,  the  CPIF  is  the  only  contract  type  exhibiting  an  average 
overrun  ratio  after  correction  for  modifications.  Further,  the  FPI  is  slightly 
skewed  to  the  left  (mean  less  than  modal  value)  and  Is  ax>re  peaked  than  the  CPIF. 

N i .1.  J ' 

Regressions  and  Correlations  r.'*  w * 

In  order  to  gain  additional  insight  with  respect  to  strength  of  association 
among  variables  In  the  data  base,  several  step-wise  multiple  linear  regressions 
were  run  on  the  data.  Dependent  variables  chosen  were  contract  type  and  ratio 
of  final  to  negotiated  cost  by  contract  type.  Since  the  Army  data  are  more 
complete  with  respect  to  attributes  or  situational  variables,  these  regressions 
are  discussed  first,. to  be  followed  by  a less  comprehensive  set  of  regressions 
on  the  comparative  Army-NASA  data. 

The  first  Army  analysis  regressed  the  Contractor  code,  commodity  code,  nego- 
tiation authority,  date  of  contract,  amount  of  initial  contract  and  procurement 
type  as  independent  variables  on  contract  type  as  the  dependent  variable.  The 
procurement  type  (Research  and  Development,  Production,  Services,  or  Study) 
entered  first  with  a coefficient  of  determination  (R^)  of  11.6^  at  the  100% 
significance  level.  Steps  2,  3 and  k brought  in  the  date  of  contract  (7.15% 
added  R^),  the  commodity  (2.4%  added  R^),  and  the  contractor  (0.5%  added  R2}, 
respectively,  for  a total  of  13.6%  variation  explained  in  k steps.  Maximum 
positive  correlation  of  ••■0.3^  occurred  between  the  contract  type  and  procure- 
ment type,  as  noted  in  the  regression.  Maximum  negative  correlation  of  -0.23 
occurred  between  the  contract  type  and  date,  also  as  shown  in  the  regression. 

In  addition,  a negative  correlation  of  -0.28  was  noted  between  the  commodity 
and  the  amount  of  the  contract,  both  independent  variables  in  this  regression. 
Virtually  no  correlation  was  shown  between  the  negotiation  authority  and  the 
amount  of  the  contract.  The  output  of  this  regression  is  summarized  in  Table  1 11-3- 

The  regressions  for  FPI  contracts,  using  the  overrun/underrun  ratios  as  the 
dependent  variables,  resulted  in  only  k%  of  the  variation  in  the  ratio  of  final 
to  adjusted  initial  contract  amount  being  explained  by  procurement  type,  the 
only  entering  variable.  Using  the  ratio  of  final  to  unadjusted  initial  amount 
as  the  dependent  variable,  l3-^%  of  the  variation  was  accounted  for  by  procure- 
ment type  in  Step  I,  with  an  additional  3-2%  explained  by  the  commodity  in 
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Step  2 for  a total  of  22. bZ.  The  correlation  rvjtrix  showed  a naxlrwi  corre- 
lation of  ♦O.SS  between  the  initial  contract  amount  and  the  number  of  modifica- 
tions, a negative  correlation  of  -0.50  between  the  comnodity  and  initial  contract 
amount,  and  virtually  no  correlation  between  the  contractor  and  the  negotiation 
authority.  These  results  arc  summarized  in  Table  1 11-10. 

The  regressions  for  CPIF  contracts,  again  using  the  overrun/underrun  ratios  as 
t^  dependent  variables,  resulted  in  I7.25t  of  the  variation  in  the  ratio  of 
final  to  adjusted  cost  explained  by  the  negotiation  autliority.  Using  the  ratio 
of  final  to  Initial  cost  as  the  dependent  variable,  kJ.Sll  of  the  variation  was 
associated  with  the  number  of  modi f icat ions.  In  the  correlations  of  independent 
variables,  a ♦.9097  correlation  coefficient  was  found  between  nuober  of  modifi- 
cations and  initial  contract  amount.  Apparently  then,  for  CPIF  contracts  par- 
ticularly, the  number  of  contract  modifications  Is  strongly  associated  with 
both  overruns  and  the  total  size  of  the  contract.  Although  it  is  dangerous  to 
cause  and  effect  relationships  from  regression  results.  It  appears  that 
contracts  with  many  modifications  tend  to  lead  to  large  cost  overruns. 

The  results  of  the  CPIF  regressions  and  correlations  arc  shown  in  Table  lll-ll. 

Since  incentive  and  cost -reimbursement  contracts  are  of  principal  interest. 

Table  111-12  summarizes  the  regression  results  for  the  Army  FPI,  CPIF  and 
CPFF  contract  types.  Results  of  the  regressions  for  the  combined  sample  of 
300  Army  and  17^  NASA  contracts  as  well  as  for  the  NASA  contracts  alone  are 
- shown  in  Tables  111-13  through  111-17.  Because  none  of  the  independent  vari- 
able^ values  arc  knoMi  prior  to  negotiation  these  regressions  are  not  nearly  as 
meaningful  as  the  Amy  contract  regressions  which  included  ''a  priori**  situa- 
tional variables  such  as  the  commodity,  negetiation  authority  and  procurement 
type.  These  situational  variables  were  not  available  in  the  NASA  data  base 
as  collected. 

Distributions  of  Cost  Ratios  - ; 

As  indicated  In  the  foregoing  review,  the  literature  is  either  silent  on  or 
assumes  that  the  ratio  of  final  to  adjusted  negotiated  cost  is  normally  dis- 
tributed. The  test  of  the  hypothesis  that  this  ratio  is  appropriately  repre- 
sent^ by  a Normal  Distribution  resulted  in  rejection  at  the  .05  level  of 
significance  for  all  categories  except  for  the  small  NASA  Headquarters  sample. 

The  goodness  of  fit  test  used  was  the  Kolnogorov-Smi rnov  one-sample  test 
described  by  Clark  and  Schkade.^3  a principal  advantage  of  this  test  Is  that 
It  may  be  used  on  small  saiaples  when  the  chi-square  test  would  be  impractical 
because  of  the  requirement  that  each  expected  frequency  be  at  least  five. 

The  test  compares  the  sample  cumulative  distribution  with  the  theoretical 
distribution.  The  point  at  which  these  two  distributions  show  the  greatest 
divergence  is  measured  and  compared  against  critical  values  at  various  levels 
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of  significance.  For  sample  sizes  la»gcr  than  35f  the  critical  value  of  this 
maximuni  difference  at  the  .03  level  of  significance  is  1.36/^  n,  where  n is 
the  sample  size.  The  test  at  the  .05  level  of  significance  results  in  rejec-  > 
tion  of  the  hypothesis  that  the  sample  Is  drawn  from  a tlornal  distribution.  i 
• These  results  are  shown  in  Table  1 1 1-6.  ! 

SaiMARY  AtlO  IMPLICATIWS  V ■*  - .:,V 

'.jf'tr- 

The  preceding  discussion  Is  concerned  with  the  relationship  between  negotiated 
and  actual  costs  In  government  procurement.  The  research  focuses  on  a descrip- 
tive evaluation  of  cost  outcomes  for  contracts  from  two  governmental  agen- 
cies as  well  as  an  Inferential  analysis  of  cost  outcomes  against  a number  of 
situational  variables.  The  results  indicate  that  there  are  no  striking  differ- 
ences between  Army  and  NASA  contract  cost  outcomes,  that  cost  ratios  are  not 
normally  distributed  as  implied  in  the  literature,  and  that  somewhat  counter- 
intuitive relationships  between  situational  variables  and  both  contract  type 
as  well  as  cost  ratios  prevailed.  The  contention  in  this  discussion  is  that 
this  paper  makes  a contribution  because  it  provides  inter-agency  comparisons, 
examines  a broad  set  of  variables  selected  from  a random  sample  of  contracts, 
and  focuses  on  the  general  behavior  of  costs  from  an  objective,  descriptive, 
statistical  point  of  view*  Finally,  the  research  Is  anchored  in  actual  cost 
behavior  rather  than  the  opinion  statement,  untested  theory,  or  "horrible 
example"  analysis  which  has  come  to  represent  Mch  of  what  procurement  re- 
search  Is  today..  ~ 
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During  the  1920'n,  two  scaclstlclana,  Saynond  Pearl  and  L.J. 

-*,  • r *****’  ***”.  «'*«“*vely  involved  in  research  dealing  with  forecasting 
®f  population  and  biological  growth.  They  discovered  that  the 
j'  ,.  S-«haped  curve  for*  of  Figure  1 accurately  represented  growth  of 

' ■ ' .!  .'•>  ■ ■ ■ • . . . I - 

. ‘•‘f*  type  which  is  characterized  by  three  distinct  phases:  (1)  a - 

slow  i^riod  of  developnent,  (2)  a rapid  period  of  expansion,  and 

(3)  a tapering  off  at  Mturity. 
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Figure  1.  The  Growth  Cur^e 
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The  growth  rates  of  a nuoiber  of  developing  industries  were  also 
found  to  follow  this  general  fonaat»  but  oost  acteapts  to 
use  this  curve  fora  for  forecasting  %rere  non-productive • The 
failure  of  the  two  forms  of  the  S-shaped  curve  (the  Logistic  and 
the  Goapertx)  (See  Appendix  1)  to  provide  accurate  forecasts  can  be 
attributed  to  two  principle  causes: 

(1)  The  equation  for  the  Gonperts  curve  assumes  nyaactry 
around  an  inflection  point  which  is  the  geometric  aean  of  the  T ^ 
values,  while  the  equation  for  the  Logistics  curve  makes  the  same 
assuaption  for  the  true  aean  of  the  T values.  Neither  of  these 
assuaptions  is  usually  justified  in  real-world  situations.^  : 

.(2)  Because  of  the  order  of  the  polynoaials  needed  to  express 
these  curves,  slight  perturbations  of  the  data  early  in  the  growth 
proc'fss  can  quickly  force  the  equation  for  either  of  the  curyes  to 
forecast  an  unrealistically  high  or  low  figure  for  total  growth.  * 

This  failure  to  accurately  forecast  becomes  iaportant  in  the 
context  of  Research  and  Developaent  within  the  Department  of  Defense 
because  R & D prograas  also  follow  the  S-shaped  curve  form  in  terms  of 
both  cuaulative  dollar  expenditures  and  cumulative  milestone  com- 
pletion over  time.  Tills  fact  should  be  no  surprise  since  the 
RAD  process  is  intuitively  a biological  growth  process  in  which 
there  is  slow  initial  development  of  the  specific  project,  followed 
by  a fairly  rapid  building  phase  which,  in  turn,  is  follo«fed  by  a 
tapering-off  to  completion. 

^Table  VI  shows  the  actual  inflection  point  for  21  weapon  systeiss 
Note  that  in  no  case  docs  this  point  consistently  fall  anywhere  close 
to  either  the  true  or  the  geometric  mean* 
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The  existence  of  these  three  phases  in  Department  of  Defense 
R & D programs  was  verified  by  taking  the  Cost  Performance  Reports 
(CPRs)  for  22  different  weapon  and  component  systems  and  tracing 
the  actual  costs  as  they  were  Incurred  throughout  the  life  of  each 
R & D program.  Ufthout  exception  the  S-shaped  curve  was  the 
dominating  factor  in  the  system  development  as  it  actually  — i:. 
occurred.  The  milestones  associated  with  these  program  also 
exhibited  the  same  characteristics  in  every  case  for  which  data 
was  available.  ..  ‘rd.* 

The  results  based  on  the  original  22  weapon  systems  were 
then  checked  against  new  and  very  complete  which  was  provided 

by  another  source  cm  IS  weapon  systems.  Again  the  connection 
between  RAD  expenditures*  the  S^shaped  curve*  and  milestone 
completion  were  firmly  established. 

Conventional  wisdom  would  suggest  that  since  the  Logistics 
and  Gompertx  curve  forms  are  generally  non-productive  forecasting 
tools*  one  should  proceed  instead  to  construct  an  econometric 
model  of  the  R & D process.  This  model*  which  would  embody  each 
of  the  endogenous  variables  in  any  given  RAD  project*  could 
then  use  the  historical  cost  data  for  each  of  these  inputs  (labor* 
raw  materials*  etc.)  to  establish  trends.  The  trends  could  then 
be  coihined  to  provide  an  overall  forecast  of  the  final  cost  of 
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Unfortunately,  the  existing  data  concerning  the  prices  of 
■any  raw  aaterlals  such  as  aluainua,  stainless  steel,  titanita, 
etc*,  are  either  Mis leading,  erroneous,  or  simply 
reflective  of  the  true  prices  actually  paid.  In  addition, 
labor  data  is  often  not  available  for  the  narrow  categories  of 
labor  actually  employed  in  a given  R t D project,  nor  does  it 
properly  reflect  local  or  regional  conditions  which  nay  significantly 

- ■ ' • * J.  » jCJi-ti-T-  _•  -..'.rt-,; 

affect  wage  rates. 

n;-.-  C-.  .1..  • , . r...  -*  t.-.rr.  ( , , : . 

These  deficiencies  rule  out  the  use  of  conventional  econometric 
modeling  techniques,  and  one  is  faced  with  the  problem  of  finding 
some  known  element  coMon  to  all  R i D prograM  which  could  be 
adapted  as  a forecasting  tool. 

Since  the  S-shaped  curve  is  a comwn  element  in  all  R 4 D 
projects,  this  paper  will  deal  with  an  effort  to  modify  the  curve 
in  such  a way  that  accurate  forecasting  can  be  achieved.  ' 
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CHAPTER  I 
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■»'>  » . • ' . i'*  /lA  ,.  , A*, 

FITTIJiC  THE  CURVE  'T  V 

► :t’«* . r:  ^ : r.  i.  . s'rir  " 

,i  The  major  problem  in  handling  an  S-shaped  growth  curve  is  one 

► ■ ^ ^ . - / * '/^  , i<7  ?.  i:k% 

of  keeping  the  higber-*order  polynomials  necessary  to  express  this 

*r  r . r • t:  , ‘ :•i^ 

complicated  curve  form  under  control.  If  one  realizes  that  any 

vi*  * J ^ f ' \ t . -»;  ? h.^i{y^r 

S-shaped  curve  is  merely  the  cumulative  form  of  a bell  curve  (which 

may  or  may  not  be  skewed)  as  shown  in  Figure  2,  a solution  to  this 
particular  problem  becomes  apparent.  •' 
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Figure  2.  The  Derivation  of  the  S-shaped  Curve 
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The  S-shaped  curve  may  be  separated  (or  •broken”)  at  the 

iaflection  point  to  yield  two  simpler  curves  of  a form  which  may 

be  expressed  by  either  a logrithaic  (y  • ax^)  or  a quadratic 

2 

ty  * * ^*2^  *2,  ^ ^2  *1^  equation.  Standard  econonetrlc  fitting 

techniques  may  then  be  utilized  to  detendne  which  of  these  two 
curve  forms  is  the  best  fit. 

^ However  g the  selection  of  the  best  curve  form  is  greatly 
complicated  by  the  fact  that  the  time  dependent  nature  of  these 
curves 9 and  indeed  the  entire  R & D process,  tends  to  cause  a 
great  deal  of  autocorrelation  in  the  resulting  data.  This  auto- 
**hich  is  a violation  of  the  necessary  assumption  that 
regression  residuals  are  not  related,  must  be  dealt  with  and 
eliminated  from  the  data  before  a valid  selection  of  equations  can  ‘ 
be  made  to  express  the  lower  and  upper  halves  of  the  S-shaped  curve. 
For  this  particular  research,  autocorrelation  was  removed  through 
the  use  of  Generalized  Uast  Squares  following  an  autoregressive 
transformation  which  utilized  a two-step  procedure  for  the  estimation 
of  Rho.  This  process  is  explained  la  its  entirety  In  Appendix  2. 

It  should  be  noted  that  the  treatment  of  each  additional 
order  of  autocorrelation  by  this  method  removes  one  additional 
data  point  from  the  lower  end  of  whichever  half  of  the  S-shaped 
curve  is  being  ‘considered.  This  occurs  because  each  succeeding 
transformation  steps  back  in  time  one  data  point  further  in  an 
attempt  to  remove  any  past  Influences  on  the  current  regression 
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residual.  This  process  has  an  inherent  benefit  in  that  it  tends 
to  weight  more  recent  observations  more  heavily  than  past  observa- 
tions* The  result  is  a rather  sophisticated  smoothing  technique 
which  may  be  varied  according  to  the  user *8  desires  to  assure  that 
the  fitted  curves  will  converge  rapidly  on  the  proper  values* 

Tables  1 through  IV  show  the  results  of  this  curve  fitting 
technique  when  it  was  applied  to  both  actual  and  budgeted  cost 

j 

figures  for  22  aejor  veapon  systeas  or  R & D projects.  In  slaost 

4 

every  caseg  the  quadratic  equation  provided  the  beat  fit  of  both 
the  upper  and  lower  segments  of  the  S-shaped  curve.  The  statistics 
accompanying  each  of  the  curve  equations  indicate  that  the  upper 
and  lower  curve  segments g when  rejoined,  do  provide  an  excellent 
proxy  for  the  original  S-shaped  curve  for  each  R & D project*  and 
the  mean  square  error  figures  accompanying  each  equation  assure 
that  these  reconstructed  S-shaped  curves  should  be  useful  fore- 
casting tools.  ' -I  t.-"j  ±,-  , 

vith  the  curves  for  each  of  the  specific  programs  developed, 
the  same  technique  of  calculating  the  upper  and  lower  halves  of  the 
S-shaped  curve  and  then  mating  these  segments  at  the  inflection 
point  can  be  used  to  derive  a general  curve  based  on  all  weapon 
systems  for  which  data  is  available.  This  general  curve  can  then 
serve  as  the  forecasting  vehicle  for  new  weapon  systems. 

The  actual  calculations  for  the  general  curve  are  accomplished 
by  pooling  all  of  the  data  for  17  systems.  Table  VI  shows  that  the 
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Dean  inflection  point  for  these  systems  occurs  at  the  .5621  of 
expenditures  end  .4622  of  time  points  with  the  width  of  one  standard 
deviation  around  this  point  being  .0342  for  expenditures  and  .0732 
for  tine.  Considering  that  these  figures  were  derived  froa  a ^ ^ 
collection  of  systevs  which  were  so  diverse  as  to  include  an  early 

«»*  I 

1930*s  era  weapon  aystea  (the  F*103),  cargo  and  boaber  aircraft  (C-5» 

s i * 

C-1419  0-1,  etc.) 9 engines  and  guns  (AnlO)*  and  planned  projects 

.j 

such  as  space  tugs^  these  figures  represent  a surprisingly  narrow 

confidence  interval. 

i 

Exhibits  1 and  II  show  the  results  of  fitting  the  curve 
segments  using  a qaadratic  equation  fo?  both  the  lower  and  upper 
halves.*  A certain  degree  of  heteroskedasticity  was  introduced 

r 

into  these  data  sets  by  a noraalization  of  the  data  which  forced 
the  S-shaped  curve  to  begin  at  the  02  tine  and  expenditures  point 

f 

and  end  at  the  1002  tine  and  expenditures  point.  Because  of  the 
obvious  significance  of  the  data  provided  in  Exhibits  1 and  II, 
the  heteroskedasticity  was  not  deemed  to  be  a significant  problen. 

Figure  3 shows  the  general  curve  fom  and  a lo  confidence 
interval  around  this  curve.  The  calculations  for  this  confidence 
interval  nay  be  found  in  Table  VI.  This  type  of  curve  derivation 
has  the  obvious  advantages  of  generating  a result  which  should  have 
wide  application  across  the  broad  spectrun  of  weapon  systems , and 
as  such  it  will  fom  the  basis  for  the  forecasting  efforts  which 
will  be  explored  in  the  next  chapters. 
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CHAPTER  II 


IHE  RATIONALE  BEHIND  THE  USE 

-r  r • T f. 

OF  THE  S-SHAFED  CURVE 

^ ^ V • I w ^ 

Chapter  I shoved  that  the  S-shaped  curve  described  every 
■illtary  RAD  project  %ihich  was  investigated  and  this  chapter 
deaonstrated  a practical  aethod  of  finding  the  curve  for  either 
a specific  project  or  a general  aodel«  However,  the  rationale 
behind  the  use  of  both  a two-part  curve  and  the  initial  budget 

r #iJJ 

data  in  the  forecasting  process  deserves  further  explanation. 

Since  no  one  can  estinate  the  cost  to  develop  a new  weapon 
system  without  an  effort  which  would  virtually  duplicate  the 
original  contract-letting  process,  it  is  obvious  tiiat  the  rational 
approach  is  to  accept  the  proposed  R & D budget  as  decided  by  the 
contractor  as  the  best  initial  estimate  of  the  project  cost. 

However,  once  this  approach  is  decided  upon  and  it  is  realised 
that  a two-part  S-shaped  growth  curve  can  be  chosen  to  express  the 
program  as  a whole,  three  very  interesting  approaches  may  be 

1 

adopted: 

(1)  The  proposed  project  budget  may  be  compared  to  all  past 
R & D budgets  to  see  if  the  expenditure  pattern  is  generally  rational 
in  light  of  R A D experience  with  other  weapon  systems.  Figure  3 
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shows  the  general  curve  with  a lo  confidence  band  based  on  the  R g D 
project,  which  were  studied  In  this  paper.  Considering  the  diversity 
of  Che  weapon  systems  which  were  Investigated,  this  band  represents 
. fairly  narrow  range  within  which  the  expenditure  pattern  for  any  • 
new  R « 0 project  should  be  expected  Co  faU.  If  the  proposed  ' 
expenditure  pattern  fall,  outside  of  this  range,  the  iontwctor 

should  be  expected  tc  Justify  why  his  program  is 'unique  in  this  ^ 
respects  ^ ^ * c ? ,5 

(2)  The  proposed  budget  expenditure  pattern  can  be  colpar^  * 
to  the  proposed  project  milestones  to  check  the  s^dfic  rationjaity  - 
of  the  budget  proposal.  Table  V shows  a reperesentative  selection  . 
of  projects  for  which  ■ilescone  data  is  available.  There  is  a very 
high  order  of  correlation  between  the  budget  expenditure  pattern  ' 
and  the  pattern  in  which  the  milestones  are  accomplished.  This 
suggests  that  the  manner  in  which  the  milestones  are  completed  <!ver 
time  provides  an  excellent  cross-check  on  the  way  in  which  the  budget 
should  be  expended.  And  it  also  means  that  slips  or  changes  in  the 
order  of  milestone  completion  will  Invariably  be  reflected  in  the 
expenditures  for  the  entire  project. 

(3)  When  the  contractor  or  procurement  officer  considers 
uncertainty  in  any  R & D program,  he  can  generally  resolve  this 
uncertainty  into  different  types  of  unknowns.  Drake  states  that 

these  unknowns  are  of  two  kinds: 

(1)  The  unknowns  that  he  Is  aware  of  and 
^®^f®ves  he  can  resolve  when  he  accepts  a 
contract— for  example,  the  configuration  of 
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an  aircraft's  slat8»  flaps,  thrust  reversers, 
speed  brakes,  and  other  devices  needed  to  meet 
the  specified  performance  factors  (take-off, 
landing  distance,  etc.).  ^ 

(2)  The  unknowns  which  are  bound  to  crop  up 
unexpectedly  and  for  which  he  is  not  prepared — 
the  'unanticipated  unknowns.*  Strictly  speaklM, 
they  are  the  unknowns  that  ocamot  be  foreseen.^ 

In  other  words,  type  1 unknowns  are  known  unknowns  while  type  2 


i\A7i 


unknowns  are  unknown  tsiknowns. 


If  one  then  pictures  hinself  standing  at  the  beginning  of  an 

' ' ' * 4 

t t D project  and  looking  down  the  tins  line  toward  the  cowpletlon. 
of  the  project,  he  will  see  the  two  different  types  of  ^uncertainty 
shown  in  Figures 
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Figure  4.  Uncertainty  Due  to  Known  Unknowns 


^Drake,  Hudson  B.,  "Major  DoD  Procurements  at  War  with  Reality," 
The  Harvard  Business  Review,  January- February  1970,  p.  124. 
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Figure  5.  DncerCalnty  IXie  to  Unknown  Unknowns  I 
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These  two  figures  can  be  conblned  to  show  that  the  total  . 
progran  uncertainty  Is  really  a kinked  curve  as  shown  In  Figure  6. 
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Figure  6.  Total  Prograa  Uncertainty 
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The  critical  poiot  is  the  inflection  point  of  the  program  since  it 
is  at  this  point  that  most  of  the  uncertainty  has  been  dispelled. 
Thus,  accurate  forecasting  la  most  essential  and  also  aost  difficult 
in  the  early  part  of  the  R & D program.  The  two-part  curve  develop- 
ment overcomes  the  problem  of  changing  uncertainty  by  allowing  the 
researcher  to  forecast  a new  lower  curve  and  inflection  point  from  ^ 
the  actual  expenditures  in  the  program  being  investigated. 

I 

This  forec*.  t may  then  be  constrained  by  using  the  top  half 
of  either  the  budget  or  the  general  curve  in  the  area  where  uncer- 
tainty is  lower  and  hence,  these  curves  should  be  the  most  accurate 
estimators  of  program  performance. 

It  should  be  assumed  that  since  the  contractor  and  the 
government  have  both  attempted  to  plan  the  R & D project  in  the 
best  possible  manner,  this  planning  effort  should  not  be  %fholly 

I 

disregarded  as  soon  as  the  first  actuals  begin  to  appear.  Instead, 

V 

it  is  more  rational  to  let  the  general  frame%#ork  of  the  expenditure 
pattern,  as  expressed  in  the  budget  curve,  continue  to  represent 
this  planning  as  an  inherent  constraint  during  any  forecasting 
activity.  Used  in  this  manner,  the  original  budget  expenditure 
curve  can  be  viewed  as  a storehouse  of  subjective  or  Judgmental 
information  which  may  be  used  to  constrain  a forecast  and  thus 
allow  for  future  inputs  by  the  managers  in  control  of  the  R & D 


program. 


These  three  points  indicate  why  the  use  of  the  two  part 
S-shaped  curve  is  particularly  appealing.  Not  only  is  the  contractor 
subjected  to  Intense  scrutiny  during  the  initial  planning  phase  . ,,!•*[ 
(points  one  and  two) , but  once  be  passes  these  checks  his  planning 
■sy  then  be  used  as  a forecasting  constraint  during  the  n*t  of 
the  prograa 
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DEVELOPING  AND  USING  THE  S-SHAPED  CURVE  r 

The  general  aetbod  for  the  developaent  and  use  of  the  S>shaped 
curve  for  a apecific  R & D prograa  follows  these  steps: 

, X...  The  budget  figures  for  the  RAD  prograa  are  gathered 
and  the  aonthly,  quarterly,  or  other  increaental  expenditures  are 
recorded  as  a cianilatlve  percentage  of  the  total  expenditure. 
Similarly,  the  amount  of  tli:3  over  which  the  prograa  Is  to  run  Is 
determined  and  each  succeeding  Increment  Is  recorded  as  a cwulatlve 
percent  of  the  total  prograa  tlae.  The  tlae  Increments  and  budget 
Increments  must  be  the  same.  This  step  has  the  effect  of  normalizing 
the  prograa  so  that  It  can  be  compared  with  all  past  RAD  pro^t 
when  plotted  on  the  axis  of  Figure  7. 
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Figure  7*  Expenditures  vs.  Ti»e 
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II.  With  the  data  arrayed  in  this  nomalized  format  and 
plotted  on  the  axis  of  Figure  7^  the  budget  expenditure  pattern  may 
be  immediately  checked  for  general  rationality.  This  is  accomplished 
by  determining  whether  or  not  the  proposed  budget  curve  falls 
within  the  confidence  band  which  has  been  constructed  for 

past  R & D projects  (Figure  3).  If  the  proposed  budget  expenditure 
pattern  falls  within  this  band»  the  R & D project  planning  is  in 
accordance  with  past  R & D experience.  However » if  the  expenditure 
pattern  falls  outside  the  confidence  band  it  should  be  determined  * 
why  this  particular  program  is  planned  to  proceed  differently  than 
all  past  programs  have  proceeded.^  • • > ' -•  j.j 

III.  The  total  number  of  milestones  connected  with  all  phases 
of  the  R & D project  is  determined  and  the  month  in  which  the  end 
or  completion  point  of  each  milestone  occurs  is  noted.  The  cumula- 
tive percentage  of  completed  milestones  is  then  calculated  for  each 
period  for  which  budget  data  is  available.  This  step  has  the  effect 
of  putting  milestone  accomplishment  in  the  same  normalized  form  as 
the  financial  data.^  This  normalized  milestone  accomplishment  data 


At  first  it  may  seem  counterintuitive  to  sum  up  milestones 
in  the  same  manner  in  which  one  sums  up  dollars.  However,  even 
though  it  is  obvious  that  no  two  milestones  represent  identical 
amounts  of  accomplishment,  deriving  cumulative  stss  of  milestone 
completions  by  financial  period  has  invariably  provided  an  excellent 
picture  of  the  manner  in  which  the  program  is  expected  to  proceed* 

It  should  be  noted  tliat  the  best  results  have  been  obtained  when 
the  milestones  were  expressed  on  the  lowest,  or  most  specific  level 
available.  Every  attempt  should  be  made  in  this  process  to  avoid 
double  counting  between  specific  and  general  program  milestones,  al- 
though no  adverse  affects  have  been  noted  when  different  specific  and 
general  milestones  were  summed  together  using  this  technique. 
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is  then  regressed  with  the  normalized  budget  data.  The  results  of 
this  regression  should  show  an  extremely  significant  relationship 
between  the  budget  expenditures  and  the  milestone  co^>letion. 

Table  V shows  the  results  for  five  sample  programs  to  give  a general 
idea  of  the  significance  of  the  relationship.  If  this  relationship 
does  not  appear,  the  proposed  expenditure  pattern  for  tne  specific 
program  in  question  is  highly  dubious  and  indicates  an  unrealistic 
program  plan  which  could  easily  lead  to  financial  problew. 

IV.  Next  locate  the  largest  Incremental  change  in  budgeted 
expenditures  which  is  followed  by  two  periods  of  decreasing 
expenditures.  This  increment  is  designated  as  the  inflection  point.. 
The  S-8haped  curve  is  broken  at  this  point  and  the  inflection  point 
becoiiea  the  last  data  point  in  the  first  (or  lower)  curve  and  the 

data  point  on  the  second  (upper)  curves  This  comDon  point 
allows  the  curves  to  be  spliced  again  after  curve  fittings  Table  VI 
shows  the  location  of  the  inflection  points  for  the  R & D prograas 
included  in  the  initial  part  of  this  studys  Rote  the  high  degree 
of  variability  in  the  point  locations. 

Vs  Equations  for  the  lower  and  upper  portion  of  the  budget 
curve  are  developed  by  the  regression  scheme  discussed  in  Appendix  2 
Due  to  the  shape  of  the  curves,  a quadratic  (y  ■ a + b^^  Xj^  + b2  x^) 
or  logrithmic  (y  ■ ax^)  curve  equation  is  usually  appropriate. 
Particular  care  must  be  taken  in  this  step  to  assure  that  the  curve 
equations  which  are  developed  have  dealt  with  the  problems  inherent 
in  the  use  of  time  series  data.  Failure  to  correct  the  problem  of 
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autocorrelation  will  result  in  curve  equations  %rfiich  are  of  little 
value  and  which  will  adversely  affect  the  perfomance  of  the 
completed  nodel.  7o  avoid  this  situation  some  techniques  of  the 
type  explained  in  the  appendix  must  be  used  at  this  Juncture » and 
indeed,  should  be  used  in  any  R 4 D forecasting. 

VI.  Once  the  curve  equations  have  been  developed  from  the 
budget  data,  two  specific  types  of  knowledge  have  been  gained: 

^ A. . The  equation  fom  which  best  fits  the  R 4 D budget 
data  has  now  been  determir2d.  This  is  usually  a quadratic  fom  for 
both  the  upper  and  lower  halves  of  the  S-shaped  curve  and  this 

specific  curve  fora  should  be  used  with  any  actuals  when  later 

* \ 

atteapts  are  made  to  forecast  the  end  price  of  the  R 4 D project. 

B.  Equations  expressing  the  subjective  planning  inherent 
in  the  R 4 D program  are  now  available  for  the  upper  and  lower  parts 
of  the  S-shaped  curve.  These  original  equations  can  be  used  as 
constraints  during  forecasting,  thus  providing  a method  of 
incorporating  this  subjective  information  into  the  final  price 
forecast.  The  specific  met’liods  by  %ihich  the  two  part  S-shaped 
curve  may  be  used  for  price  forecasting  are  the  subject  of  the  next 
chapter. 
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(t)  the  best  possible  prograa  cost,  (b)  the  most  likely  program 
cost,  and  (c)  the  worst  possible  program  cost. 

The  best  possible  program  cost  Is  the  cost  which  would  be 
incurred  by  the  R & D program  If  It  is  assumed  that  the  second  half 
of  the  program  will  follow  exactly  the  proposed  budget  vurve 
Irrespective  of  the  performance  record  established  in  the  first 
half  of  the  program.  ” 

The  most  likely  program  cost  is  that  figure  which  woiild  be 
obtained  if  the  second  half  of  the  program  follows  the  course 
indicated  by  the  general  R & 0 curve.  This  gener^  curve  being 
*^ther  one  developed  for  all  weapon  systems  such  as  that  curve 
displayed  in  Figure  3,  or  a curve  developed  for  weapon  systems  of 
the  specific  type  of  the  R ft  D project  being  investigated.  In 
other  words,  a curve  based  on  general  missile  system  data  would  be 
used  tihen  the  R ft  D project  is  for  a missile.  Intuitively,  the' 

latter  approach  should  provide  a tighter  confidence  interval  for 

» *- 

the  forecast. 

The  worst  possible  program  cost  would  be  the  figure  indicated 
by  the  upper  limit  of  the  confidence  interval  around  the  forecast. 
The  second  half  of  the  general  curve  would  again  be  used,  and  the 
confidence  interval  around  this  curve  would  provide  the  upper  limit 
(within  a certain  probability)  of  the  R & D cost. 

These  three  types  of  forecasts  are  shown  in  Figure  8. 
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Figure  8.  Tliree  Possible  Forecasts 
froB  Prograa  Actuals 


The  specifics  involved  in  foning  each  of  these  forecasts 
vill  nov  be  covered. 

THE  BEST  POSSIBLE  COST 

(1)  First  derive  the  tw  halves  of  the  equation  for  the 
S-shaped  curve  in  the  manner  previously  outlined.  This  gives 
curve  1 of  Figure  9,  the  budget  curve. 

(2)  Assume  nou  that  the  first  data  points  concerning 
actual  expenditure  information  have  become  available.  These  data 
points  arc  first  deflated  by  dividing  the  dollar  figures  by  an 
appropriate  index.  Studies  have  shown  that  the  GNP  Deflator  is 
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usually  a good  choice  for  this  index. ^ The  deflated  figures  are 
then  converted  to  percentage  figures  by  dividing  by  the  latest 
^ total  prograa  cost»  and  these  percentage  figures  are 

; plotted  on  the  axis  of  Figure  7.  This  leads  to  the  beginning  of 

an  ••actuals"  curve,  shown  as  curve  2 of  Figure  9.  These  actuals 
■ay  be  used  to  forecast  a new  end  cost  for  the  program  as  follows: 

(a)  Derive  a new  lower  half  of  the  S-shaped  curve 
by  fitting  the  actuals  to  an  equation  of  the  form  found  to  be 
appropriate  for  tlie  budget  data-in  general,  this  will  bfi  a 
quadratic  curve.  j 

" ' - . — , ' ‘j. 

. ^ 0>)  Using  this  quadratic  curve  equation,  insert 

the  percent  of  total  time  figure  for  the  budget  curve  Inflection 

point  (35Z  on  Figure  9)  to  forecast  a new  inflection  point, 

(Point  5 on  Figure  9)  and  then  use  other  points  on  the  X (time) 

« t 

axis  to  derive  a new  lower  half  for  the  S-shaped  curve. 

, (c)  Now  take  the  equation  which  was  developed  for 

the  top  half  of  the  budget  curve  and  substitute  the  percent  time 
and  percent  budget  figures  for  the  forecast  inflection  point  into 
this  equation  to  calculate  a new  intercept  for  the  upper  curve. 
This  new  intercept,  along  with  the  original  slope  figures  from  the 
budget  curve,  has  the  effect  of  ••splicing^*  the  equation  developed 

' Brush,  John  S.,  ••Study  of  Possible  Inprovenents  in  the 
Accuracy  of  Aeronautical  Economic  Escalation  Indices, ••  unpublished 
paper,  USAF  Academy,  Colorado,  February  1976. 


from  the  first  half  actuals  to  the  budget  equation  for  the  second 
half  of  the  curve,  all  of  which  yields  the  new  S-shaped  curve  4 
of  Figure  9.  In  addition,  this  procedure  allo:.*8  the  development 
of  a forecast  for  the  end  cost  of  the  project  which  is  constrained 
by  the  planning  and  other  sid>Jectlve  inforaation  Inherent  in  the  ' 
original  budget  curve.  ■ j tv  ciji-f  .03. 
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(d)  At  this  point,  one  may  take  several  different 
First,  if  it  is  desired  to  learn  the  absolute  figure 


approaches 

for  the  final  cost  of  the  project,  curve  4 nay  be  modified  by 


inclusion  of  inflation  data.  In  this  case,  the  forecast  expenditure 
data  of  curve  4 would  be  nultiplled  by  an  inflation  index  to  get  a 
new  curve  which  is  labeled  5 on  Figure  9.  However,  in  doing  this 
one  should  have  in  mind  a concept  of  the  errors  inherent  in  any 
process  such  as  the  one  Just  described.  *4^. 

Up  to  this  point  ve  have  not  Mentioned,  for  the  sake  of  • • 
siMplicity,  that  there  is  an  error  involved  in  forecasting  which 

^ . .T 

should  be  expressed  as  a confidence  interval  around  curve  4.  The 
confidence  band  indicates  that,  with  some  given  probability,  one 
■ay  expect  the  real  value  for  any  point  on  the  line  to  fall  soaewbere 

s . * 

within  this  particular  interval*  When  the  budget  curve  is  coopared 

• o 

with  the  forecast  curve,  only  one  error,  the  standard  error  of  the 
forecast,  nust  be  considered*  IMs  leads  to  the  situation  shown 


in  Figure  10 
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Figure  10.  The  Error  of  the  Forecast 


Here  Che  confidence  band  indicates  the  possible  range  of  values 


(fro*  b to  c)  in  which  the  true,  deflated  cost  of  the  prograa  is 


expected  to  fall,  and  siBilarly,  the  range  of  the  sixe  [fro*  ab  to 
ac]  of  Che  potential  prograa  overrun.  \ 

However,  if  one  desires  to  coapare  the  full  cost,  with 
inflation,  of  the  project  (5,  Figure  9)  with  the  full  inflated 
cost  of  the  budget,  both  the  error  of  the  forecast  and  the  error 
involved  in  developing  the  inflation  figures  cust  be  considered. 


S This  has  the  effect  of  greatly  increasing  the  size  of  the  confidence 


bands  as  is  shorn  in  Figure  11.  The  end  result  is  that  the  ability 
to  conpare  the  final  cost  of  the  project  with  the  budget  cost  is 


greatly  impaired.  As  Figure  11  shovs,  in  this  case  one  could 
anticipate  a treaendous  overrun  [a  - d]  and  an  underrun  [c  - bj 

frora  the  sane  data. 
tTbrac 


%rofSL  rim 


L,' 


Figure  11.  The  Error  of  the  Forecast  and 

The  Error  of  the  Inflation  Forecast 
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The  lesson  here  is  to  conpare  figures  in  a Manner  which  will 
■IniMize  the  errors  Involved  in  the  coaparison.  In  other  words, 
the  best  picture  of  the  status  of  a project  uay  be  gained  by  com- 
paring the  two  deflated  curves  shown  in  Figure  10.  This  comparison 
provides  all  of  the  information  required  for  day-to-day  management 
of  the  program.  If  a full  end  cost  of  the  program  is  desired,  this 

s'* 

can  be  developed  quickly  by  simple  multiplication  utilizing  whatever 

* • • 4^ . 7/ 

InflatloD  forecast  is  deened  appropriate  at  the  tiae  that  the 
iDforaatlon  is  required.  * " • 

' ' ' ■ ’ . • ' ' T-:  ..  ?- 

This  does  not  mean,  however^  that  one  should  not  use  the 
actual  inflation  data  when  it  is  available.  In  this  cas^,  no  errors 
of  forecast  are  present  because  the  actuals  in  both  prograa  cost 
and  inflation  rates  are  known.  This  makes  it  very  easy  to  reMve 
the  effects  of  inflation  to  see  how  natch  cf  an  overrun  is  actually 
attributable  to  the  contractor* 

Figure  12  shows  a case  in  which  the  deflated  budget  curve  1 
is  modified  by  the  actual  experienced  inflation  to  derive  curve  2. 
This  curve  may  be  readily  compared  with  the  contractors  inflated 
actuals  (curve  3)  to  determine  the  actual  extent  of  the  overrun 
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Figure  12.  The  Ose  of  Actual  Inflation  Data  , 

* 

(e)  Another  situation  which  this  nethod  of 
pro'grau  aonltorlng  will  easily  handle  is  the  case  of  the  schedule 
slippage  or  program  extension.  Of  the  two,  the  slippage  is  the 
Bost  severe  because  it  often  occurs  early  in  the  project  where  it 
has  a profound  effect  on  costs.  Assum  once  again  the  basic  de- 
bated budget  curve  shown  in  Figure  13  with  an  actual  deflated 
expenditure  curve  as  shown. 
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Figure  13.  The  Program  Slippage  Situation 
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It  would  appear  upon  Initial  inspection  that  the  prograa  is  running 
slightly  below  the  planned  expenditures  at  tine  t.  However,  it  is 
revealed  that  the  R & D prograa  is  actually  behind  schedule,  having 
only  accomplished  the  number  of  sd.lestones  associated  with  tine  t-1. 
To  compensate  for  this  slippage,  curve  2 is  moved  back  one  unit  from 
t to  t-1  so  that  the  actual  expenditures  are  now  shown  as  curve  3 in 
their  proper  relationship  with  the  budget  curve.  This  is  actually 
accomplished  mathematically  by  calculating  a new  inflection  point 
which  wUl  reflect  the  slippage  in  the  schedule.  This  new  point  is 
derived  from  the  equation  for  curve  2 by  calculating  the  Inflection 
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point  not  at  time  Tj  the  location  of  the  original  point,  but 
rather  at  tioe  T + 1,  the  location  of  the  inflection  point  after 
slippage  has  occurred.  This  new  inflection  point  becoaes  the 
intercept  of  the  equation  for  the  top  half  of  the  budget  curve,  and 
the  tioe  values  which  are  used  to  forecast  froa  the  top  half  of  the 
budget  curve  now  start  at  the  T -I-  I Increment  (Instead  of  T)  and 

continue  to  the  lOOZ  -f  1 increment  (instead  of  the  lOOZ  increment). 

I 

: Wr*  t TOE  MOST  LIKELY  COST  - . . v :u 

’ (PORECASTINC  reib-G  THE  CEMERAL  CURVE)  ' j 

Forecasting  the  most  likely  cost  proceeds  in  the  same  manner 
listed  in  the  previous  section  up  to  the  point  at  which  a new 
inflection  point  is  forecast.  The  actuals  are  deflated,  converted 
to  percentages  and  plotted  in  the  same  manner,  and  the  curve  form 
to  plot  these  actuals  is  the  sane  equation  type  selected  to  , 
describe  the  bottom  half  of  the  general  curve.  At  this  point,  " 
however,  the  method  of  forecasting  changes  considerably. 

Instead  of  merely  splicing  the  top  half  of  the  general  curve 
ont-o  the  new  bottom  curve,  the  bottom  curve  is  actually  napped  into 
the  general  curve  framework.  This  is  accomplished  as  follows: 

(1)  Using  the  deflated  actuals  from  the  program,  fit 
whatever  curve  form  is  used  in  the  bottom  half  of  the  general 
curve  to  this  data  and  forecast  a new  value  for  cumulative  expendi- 
tures at  the  inflection  point.  This  new  value  is  found  by  substi- 
tuting the  cumulative  percent  tine  figure  which  corresponds  to  the 
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general  curve  Inflection  point  into  the  new  equation  which  was 
derived  from  the  actuals. 

(2)  Take  the  new  value  for  cumulative  expenditures 

and  let  this  value  be  equal  to  the  cumulative  percent  of  budget 
« ' 

figure  which  is  associated  with  the  inflection  point  on  the  general 
curve.  . ....  r 

(3)  Using  the  relationships  established  in  1 and  2,  the 
top  half  of  the  general  curve  may  mow  be  converted  from  cumulative 
percentage  figures  to  forecast  cumulative  expenditures  for  the 
program  being  investigated. 

. » .*.1,  . • • jriz -f. 

Thi»  forecasting  Kthod  has  several  advantages: 

. _ , (1)  The  tiae  ever  which  the  prograa  is  planned  to  run 

is  taken  as  a given  unless  evidence  to  the  contrary  is  discovered. 

(2)  The  lower  curve  forecast  is  mapped  into  the  general 
curve  formac^  thereby  creating  a smooth  S-shaped  curve  for  the 
entire  program.  Sinply  splicing  the  curves  as  is  done  with  the 
budget  curve  in  the  previous  section  will  often  create  discontinu- 
ities in  the  curve. 

(3)  The  forecast  which  is  created  in  this  method  is 
based  strictly  on  the  assumption  that  expenditures  in  this  particular 
program  are  proceeding  in  the  same  manner  that  all  past  programs 
have  proceeded. 

• ^ ^ A 

THE  WORST  POSSIBLE  COST 

Developing  the  forecast  for  the  worst  possible  cost  is  only  a 
matter  of  slightly  modifying  the  previous  most  likely  cost  forecast. 
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A confidence  interval  for  the  motit  likely  cost  is  calculated  by  tlie 
■ethods  covered  in  Chapter  V»  The  upper  limit  of  this  confidence 
band,  based  on  whatever  level  of  confidence  was  selected  by  the 
analyst^  will  ^ve  the  cost  figure  that  one  can  be  XZ  certain  will  not 
be  exceeded.  Coupled  with  the  most  likely  cost,  this  is  an  excellent 
management  tool.  i ^ /I 

In  suxmary,  three  possible  forecasts  may  be  developed  from  the 
S-shaped  curve.  The  **spliced”  curve  forecast  using  the  program 
budget  curve  reflects  an  expenditure  of  the  lowest  possible  magnitude 
which  can  be  obtained.  For  this  expenditure  to  be  realised,  the 
program  must  run  exactly  as  planned  from  the  inflection  point  onward. 
This  is  a highly  unlikely  situation  if  any  increased  expenditures 
have  been  incurred  early  in  the  program.  The  most  likely  cost  and 
its  confidence  band  which  extends  to  the  upper  confidence  limit 

t 

(or  the  worst  possible  cost)  for  the  program  are  clearly  the  most 
realistic  forecasts.  This  is  because  the  method  of  mapping  the  new 
forecast  for  the  bottom  of  the  curve  into  the  general  curve  format 
places  the  entire  program  in  a more  legitimate,  historical  perspective 

FORECASTING  IN  THE  AREA  OF  THE  INFLECTION  POINT 

With  the  general  methods  for  handling  data  early  in  program 
established,  the  next  area  of  interest  is  the  forecast  which  is  made 
when  the  string  of  actual  expenditures  stretches  all  the  way  to  the 
assumed  inflection  point  in  the  program.  When  this  takes  place  two 
courses  of  action  are  called  for  as  the  actuals  approach  and  cross 


the  inflection  point: 


(1)  Until  the  inflection  point  has  been  reached,  the  best 
nethod  of  forecasting  is  to  continue  to  develop  a new  lower  half  of 
Che  curve  tram  Che  actuals,  and  then  to  map  this  curve  into  the 
general  curve  as  was  done  In  the  previous  sections  of  this  chapter. 

<2)  After  the  actuals  appear  to  have  crossed  Che  Inflection 
point  (i.a..  a large  expenditure  In  a given  period  has  been  followed  .. 
by  two  periods  of  decreasing  expenditures)  one  can  proceed  in  two 
different  mays:  .. 

(•)  ^ actual  data  points  continue  to  fit  the 

top  half  of  the  general  curve  vithia  an  appropriate  confidence  ^ - J 

interval,  continue  to  forecast  by  using  the  top  half  of  the  general 
curve*  ..  ;•  « 

(b)  If  the  actual  data  points  are  diverging  froa 
the  general  curve,  if  enough  actual  data  points  exist  beyond 
the  inflection  point  for  regression  analysis  to  be  used,  then  calculate 
• equation  for  the  top  half  of  the  curve  using  whichever  equation 
fom  w appropriate  in  deriving  the  general  curve.  This  method 
presumes  that  at  least  5 data  points  have  occurred  past  the  inflection 
point*  It  should  be  noted  that  good  accuracy  in  generating  a new 
curve  will  not  occur  until  at  least  10  points  have  been  identified. 


A 


CONCLUSION 

. The  general  techniques  concerning  the  use  of  inflation  in  the 
model  and  the  method  of  handling  time  slippages  are  applied  to  the 
forecasts  based  on  the  general  curve  in  exactly  the  saioe  manner  as 
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they  were  used  with  the  budget  curve.  The  forecasting  method 
selected  should  be  the  one  which  best  serves  the  decision  malu^r's 
needs. 

Since  the  general  curve  would  be  representative  of  either  a 
mix.  of  weapon  systems  or  a specific  type  of  weapon  system,  either 
of  which  have  been  completed  at  some  time  in  the  past,  the  data 

4 

around  which  this  curve  is  structured  shotild  be  closely  examined. 

• ■ 1 ■ ■ • 

It  will  be  readily  apparent  in  most  cases  that  these  completed 

I . 

programs  all  contained  a certain  number  of  changes  during  their 

■ * * *'  V ' 

R b D phases.  Thus,  it  should  be  assisMd  that  the  final  cost 
figures  which  are  generated  from  forecasts  using  the  general  curve  . 
also  include  a like  nusber  of  anticipated  program  changes,  even  if 

# . r 

those  changes  are  not  visualized  at  Che  early  stage  of  the  program 
when  the  forecast  is  made. 

• . - - ■ ■ • V,  • ■ ....... 
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CHAPTER  V ’ 
AN  APPUCATION 


*>  * 


Project  B is  d weapon  system  whose  RAD  phase  started  In 
year  1.  The  level  6 budget  which  was  provided  by  the  contractor 
prior  to  the  start  of  the  R 4 0 prograa  contained  both  a total  cost 
figure  a^  Individual  figures  for  the  utegorles  of  engineering, 
■anufacturing  and  prograa  aanageaent  labor.  These  deflated  dollar 
figures  were  extracted  froa  the  budget  and  noraallsed  to  a percent 
of  total  tiae— percent  of  total  budget  basis  in  Table  VII,  The 

; 4;  t ‘-iV  ; • re-r'  * 

inflection  point  for  each  of  the  four  budget  curves  was  also 

,*  ’''tT  tT'  a. 

identified  at  this  tiae,  ' * 

The  total  expenditure  curve  waj  plotted  on  the  axis  of 
Figure  7 and  co^iared  for  general  rationality  with  the  confidence 
band  for  all  prior  R 4 D prograns  shown  In  Figure  3.  The  proposed 
budget  for  project  B was  contained  trlthln  the  confidence  band  and 
the  contractor's  proposal  was  therefore  judged  to  be  sound. 

As  a second  test,  the  total  expenditure  data  was  regressed 
against  the  project  B nilestones.  The  results  of  this  test  were 
as  follows: 

adj.  R^  - .98 
F - 165 
t - 41 


Durbin  Watson  Statistic:  1*97 
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These  s’’sclsclcs  show  a significant  relationship  between  the  adle- 
stones  and  the  expenditure  pattern,  and,  therefore,  confirm  the 
specific  rationality  of  this  particular  expenditure  pattern  for 
this  particular  project. 

. ' la  L * 

passed  these  two  important  tests,  the  budget  data  can 
be  assumed  to  be  valid  and  eiiuations  for  the  top  and  bottom  halves 
of  both  the  total  budget  curve  and  each  of  the  labor  curves  can  be 
developed.  Exhibit  HI  shows  the  output  of  the  computer  regression 
runs  for  the  total  budget  curve,  and  from  this  our,>ut  t>e  equations 
for  the  bottom  and  top  halves  of  the  curve  con  be  found  to  be 
bottom  half:  T - -.193572  + 1.884U2X^  + .470083Xj^* 
top  half:  T - .U9350  »•  1.75834X,  - .9108«,^. 

All  of  the  above  research  would  be  accoeplished.  prior  to  the  start  of 
the  program  itself,  and  th*  program  manager,  who  has  Justified  in  Itis 
own  mind  the  validity  of  the  budget  in  both  a general  and  a specific 
sense,  may  now  sit  back  and  await  tb-  arrival  of  the  actuals. 

By  the  second  quarter  of  year  three,  the  actuals  shown  in 
Table  VlII  have  been  acciasulsted.  Obviously,  the  program  manager 
would  have  been  forecasting  the  future  program  costs  each  tine  a 


^ new  set  of  actuals  arrived,  but  let  us  assume  that  this  task  is  nc<y 

to  be  accomplished  based  on  the  latest  figures  which  have  Just 
arrived.  First,  the  program  manager  fits  a quadratic  curve  form 
to  each  of  the  strings  of  actuals  representing  the  total  budget 
and  the  engineering,  manufacturing  and  program  managing  labor.  A 


( 


quadratic  is  chosen  for  the  equation  form  because,  in  each  case,* 
it  vas  this  type  of  equation  %ihlch  provided  the  best  fit  for  the 
budget  curves.  For  simplicity,  ve  will  only  deal  with  the  total 
budget  curve  in  this  example,  even  though  the  program  manager  ' 
mould  be  highly  interested  in  forecasting  each  of  the  other  three 
''curves  in  an  actual  situation.  •'*  ^*^  ^**'  .>3 

equation  for  the  total  budget  actuals  is  developed  by  a 
computer  regression  technique,  the  results  of  which  are  shown  in 
^ Exhibit  IV.  The  equation  is  ‘ ^ V ’] 

- * T - .00275322  - .0572721^  ^ 4.84246X^  . • ^ ^ 

Figure  14  shows  the  actual  expenditures  and  the  budget  curve  for 


comparison. 
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It  appears  from  Figure  14  that  the  actual  total  expenditures 
are  slightly  underrunning  the  budgeted  total  expenditures. 

However,  upon  conparlson  of  Tables  VII  and  VIII,  It  becomes  obvious 
that  some A Ing  is  amiss.  The  engineering,  program  management  and 
budget  expenditures  are  all  on  schedule  or  overrunning  while  the 


O 

U 

O 

o 


Mni^acturlng  labor  is  well  below  the  budgeted  expenditure  level. 

* 1 a*f* 

The  program  Is  obviously  not  proceeding  as  planned,  and  more  sped- 
flcally,  the  engineering  and  prograa  aanaglng  efforts  are  not 
producing  the  necessary  results  for  manufacturing  to  take  place.* 

A further  comparison  of  the  manufacturing  figures  in  Tables 
VII  and  VIII  Indicates  that  the  program  is  actually  about  1 period' 
behind  schedule  (2  quarters);^  This  means  that  a silage  has  ’ 
occurred,  and  the  actuals  curve  is  shifted  to  reflect  this  condition 


w note  that  this  condition  could  have  been  detected  at  a nuch 

earlier  time.  Manufacturing  begins  to  lag  significantly  by  the  3rd 
reporting  period. 


(_) 
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FORECAST  USING  THE  BUDGET  CURVE  I 

To  detenlne  the  effect  which  this  slippage  has  had  on  the  . * |’ 


total  budget,  we  now  forecast  the  end  cost  as  follows:  v j.;'’ 

(1)'  Forecast  a new  inflection  point  for  the  actuals  curve  by 


using  the  derived  equation  for  the  actuals  and  the  tine  period  for 


- 


quarter  2,  year  5 instead  of  the  original  inflection  point  (quarter 


0‘1 


* • . •*  jt  . r ^ 

4,  year  4)1  This  aatheBatlcally  adjusts  the  forecast  for  the  1 period  / 


slippage  noted  previously 


..  new  inflection  point 


T_-. 00275322  - .057272(.476)  + 4.84246(:476)' 


“ 1.0726779  - . ^'v  - : 


f>4  :*»T5:rrr  * rr.iJ  TI>  . 

(2)  Use  the  new  inflection  point  to  calculate  a Modified  y ) 


intercept  for  the  equation  for  the  top  half  of  the  budget  data. 


From  Exhibit  I this  curve  equation  is 


« .149330  1.75834X^  - .91088Xj^^ 


Substituting  for  Y^  yields 


o... 

1 ' . 

• : O * 


% . V 

-Xi. 


- 1.0726779  - New  Intercept  + 1.75834(.476)  -.91088(.476)“‘ 


(') 


and  1.0726779  - 1.75834(.476)  ^ .91088(.476)' 
#4420916  ■ New  Intercept 


New  Intercept 


D : 


r 

. t; 


Note  that  this  value  is  calculated  at  the  sane  inflection  point  used 


in  step  (1). 


t 


L 


■ 

f *; 


(3)  This  new  equation  for  the  upper  half  of  the  actuals 

i2  . 


Y^  - .4420916  + 1.75834X^  - .91088(X^)^ 

0 ’ 

nay  now  be  used  to  calculate  the  forecast  cost  at  any  point  from  the 
inflection  point  on  to  the  end  of  the  program.  To  calculate  the 
end  cost  of  the  R & D program  we  would  proceed  as  follows: 
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C) 


3 


/ 


. / 


(a)  With  no  slippage  the  R & D program  should  end  at 

^ . the  lOOZ  of  total  time  point.  But  with  1 period  of  slippage  the 

.V  * V«.,  S - » A 


program  should  now  end  at  the: 
od  c /(ioo  + 


u 


(100  + 5.6)Z  point 
vhere  each  tiae  Increment  has  been  equal  to 


irftC- 


f /;i  : 


5.6Z  of  the  total  time' 


''’'‘'^^  .Vend  cost  - - .4420916  + 1.75934(1.056)  - .91088(1.056)' 


f * 


JKf?  uftiij  ”-^«"li283143  ■‘iJ'fi  .40v>int  {•lity  7tiit  sjcn  oi 

or.  In  other  words,  a 1 period  slippage  in  the  program  ^ 

t 

has  induced  a 28Z  overrun  in  the  finaj.  cost. 


r Qj 


FORECAST  USING  THE  GENERAL  CURVE 

m.-^y  * 

Assuming  once  again  that  1 period  of  slippage  had  been 
, experienced  in  the  program^  the  general  curve  may  be  used  to  forecast 
a new  end  cost  as  follows:  i.  -.*  , : r;  ^ t 


(1)  The  new  Inflection  point  is  forecast  in  the  same  manner 

as  before  using  the  equation  generated  from  the  actuals  in  Exhibit  IV 

- .00275332  - .057272(.476)  + 4.84246(.476)^  ‘ 

or  T - 1.0726779  . ' 

n 

(2)  This  point  is  now  mapped  into  the  framework  of  the  general 
curve.  Figure  3 shows  that  the  inflection  point  of  the  general  curve 
is  located  at  46Z  cumulative  time  and  56Z  cumulative  expenditures. 
Thus,  if 


Y - 1.0726779,  then 


■ui— ^ ^ <•".  • 


1.0726779  - 56Z  (E) 

■**  ••Vi''  ^ I-  » 


® progra.  expenditures.  . 

. ..*.  E - (1.0726779)  T .56  - 1.9153 


. and  Che  aost  likely  total  prograa  cost  Is  forecast  to  be  191Z 
j..  of  the  budgeted  cost. 


« ^ ••  ;JL*-  .3 


' (3)  The  Method  outlined  In  (2)  can  be  used  to  fill  In  the  rest 

of  the  points  along  the  new  forecast  cost  curve,  but  It  Is  Interesting 
( to  note  that  this  forecast  calls  for  a nuch  higher  figure  cKan  that 
produced  by  the  budget  curve  "splicing"  Method.  , 


oy  tne  budget  curve  "splicing"  Method,  v ^ ri  . VV 

--  **  ■-  . ’I  ' . L . 

THE  COHFIDEKE  mEBVjU.  ‘ ^ ^ v „ 


To  use  this  forecast  in  its  point  estlaate  foraat  would  be 


**•  error.  Rather  there  Is  a confidence  Interval  around  this  fore- 
cast  which  Must  be  considered.  TOe  calculation  of  this  interval  Is 
a two-part  process  dealing  first  with  the  standard  error  of  the 


forecast  for  the  new  Inflection  point.  Exhibit  IV  shows  that 


the  variance.  Is  .000132813  for  this  particular  line.  The  standard 
*rror  of  the  forecast  Is  calculated  froa  the  formula 


Sf -Sy^yrTTTo^ 


2 


KX-X)^ 


where  X is,  in  this  case,  the  inflection  point. 


for  this  example 


S.  - .0114977/1  1 -t"  .476 

^ 5 .0339 

- .044 


and  thus  one  can  be  95Z  confident  that  the  inflection  point  is  within 
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an  interval  of 

1.0726  ^ t , S. 

— n-l  f 

or  1.0726  ± 2.776  (.044) 

1.0726  ± .1246 

Figure  16  shows  this  confidence  band  around  the  lower  half  of  the 


curve. 
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Figure  16.  The  Confidence  Band  for  the  Lower  Half 
of  the  Actuals  Curve 

When  the  top  half  of  the  curve  is  spliced  to  the  forecast  inflection 
point  an  additional  error,  namely  that  of  fitting  the  upper  half  of 
the  curve  to  the  budget  data,  is  encountered.  This  fitting  error 
can  be  read  from  Exhibit  III  " .0000271942. 

A conservative  estimate  for  the  total  error  involved  in  the 
forecast  of  the  final  cost  can  be  gained  by  conbining  the  forecast 
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• .f,  JL-. 


/ 


'J. 

error  at  the  inflection  point  with  the  fitting  error  of  the  upper  J 

half  of  the  curve  as  follows: 

' ■ 


^ • Total  standard  error  ■ 

.0440  + .0052  - .0492  - S 


I,  ■ ♦ “ 
- s 

TJ  *.  I .* 

Thus  the  confidence  band  around  the  forecast  for  the  end  cost  of  rx* 
the  prograa  using  the  budget  curve  becomes 
, 1-283143  ± t^j(S^) 

1.283143  + (2.776)(.0492) 

1.28  + .1366 


tm 

o 


--f'  -.1  . 


And.  one  can  be  952  confident  that  the  final  cost  of  program  B 
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' '•  Slailarly,  this  process  can  be  completed  for  the  forecast  made 
vlth  the  budget  curve  to  derive  the  aost  likely  cost.  Here  the 
standard  error  of  the  forecast  reMlns  . ult.v»=  .<ii<  ci-usxXij  > 

* *'  w ■<  : 

f --y  >.•»-*...  * [ >;VJv  ' .1 

as  has  previously  been  calculated.  However,  as  might  be  expected/ 

considerably  more  error  is  involved  in  fitting  the  top  half  of  the 
general  curve  than  was  present  in  the  budget  curve.  For  this  curve. 


••i 


composed  of  numerous  system  types.  Exhibit  II  shows  that  the  fitting 


error  (S^  ) is 

r * 

- .00746826 

y* 


therefore  ■ .086 

y» 


and  the  total  standard  error  (S^)  is 


. n 

.•  vt 


s - s,  + s 

t f yx 


- .044  + .086 


.13 


Mov  the  confidence  Interval  for  the  oost  likely  cost  is 


1.9153  ± 


1.9153  ± (2.776) (.13) 

1.9153 +.36 

and  there  is  95Z  confidence  that  the  nost  likely  cost  of  prograa  B is 
between  228  and  156  percent  of  the  initially  budgeted  systea  cost. 

Another  output  of  this  forecast  is  the  worst  possible  cost. 
This  cost  is  the  upper  limit  of  the  confidence  interval  or»  in  this 
case,  2282  of  the  amount  initially  budgeted  for  the  project. 

If  these  forecasts  seen  unreasonably  high,  it  should  be  noted 
that  program  B was  an  actual  program  and  that  the  final  cost  for  the 


I 

♦ , 
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R & D portion  of  program  B was  quite  close  to  the  worst  possible  cost 
(2281)  figure.  Hote  also  that  this  entire  process  has  been  accom- 
plished  with  deflated  data.  Incorporation  of  the  forecast  inflation 

N* 

fo*"  this  periml  would  have  greatly  increased  the  size  of  the 
confidence  band  in  Figure  17  and  hence,  greatly  decreased  the 
possible  forecasting  accuracy  of  the  overrun.  , ' ^ 


' • • ■ • ■ . ^ ' -■■■  ■•-'■■  Vi  ■/  A ' 'KY-' 

^ . .1 ‘ 'f  •- 

^ ' .**  * ' V ♦ • 

... 

ti*  ^ VcO. 

5’  . • ^ 

\7il  Ovil 

* f ^ I c * * 


A.-.,.-  . 


' : ; -i  • > : < X* 


/ ; . -! 


. r N - ^ V*  : . 

• ■ r -1  -i -V  ; 

J y iio  --.a.  *f  I 


94 


■V. 


. Vb  '^iuUtflav^as. 


4 >il^PP 


; I , . l-  ,11.1  VJ  IJipi|!^f9li#9!i|ppp||^ 


••-  *A  * 


1 

i 

I 

APPEBDIX  I ■ ' .; /■'  ' ■)■v  • : .■■f/'iTv': 

r vs  : ^ liWISTICS  A!ID  GOHPERTZ  CUK?B  EQUATIONS  i*.:l 

The  eq»tl<;.  for, the  Logletlc  Curve  le  . 

■ s.<».  .-k*  «.*,  -»r.  ..s 

a • (I  T - £ Y)-^^ - difference  hetwMn'  ’ V/t  i ’ >;C 


' ■■  i"; '- :. ^ 

.,5t<x  ,i  jr  J Y _ J Y --  • 7^  the  ratio  between  successive  , /7\ 

--  t *,  • . ^ ^ ..  Ancroaents  of  growth.  , VW 


^ the  difference  be^ieen'  ' 


increaents  of  growth,  i,  .:  \ ct), 

• •'  k".  A J fr  V ' h“-i  - , f- v'- 

n 'V  “tT"  ;*►■  ‘'**  “y^tote  or  liMt  /T\?^ 

;■• . .. . 

■■  •■  ■ ,^,  ■ ' • ■'■  •■  ; ^‘’:  ‘SV  :■--;  Vt.'® 

***^  ^I*  ^2*  ^3  tefer  to  the  first,  second  and  third  parts  of  Q 

the  Y values.  ■“  *•  --s..  . ,-  >- 

f ) 

a to  the  nuaber  of  observations  in  each  of  the  three  "parts" 
or  sections  of  T values. 

'Yhe  eqiiatlon  for  the  Goapertz  curve  to 


^ ^ 


bx 


using  the 


for  kg  a and  b as  shovn 


above. 
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APPCMDIX  II 

. '.*1 

REHOVniG  AUTOOOSBELATION 

> >;i  -‘V'  . .'*■•'’■.4? 

^ • * * 

Given  that  autocorrelation  exists » the  ol"icrv«^  residual  tern  2^ 

is  actually  a cofid>ination  of  two  errors 


•k  > 


^t-l 


i^re  is  the  true  error  tern 


\al 


or 


n-1 

-i-  ‘ '.Vi 

n-l  0*1 


1 

n-k 


• the  estimated  relationship 


^ *0 

o»l 


n-l 
£ 

A . ■ 

(n-l) 


V«+l 


(1) 


J 


and  e - is  the  previous  residual  terr'. 

Since  the  true  relationship  between  e^  and  unknown,  it 

must  be  estisated.  Several  methods  are  available  fcr  doing  this, 
and  this  study  employs  the  following  approach: 

(a)  coiq>ute  all  of  the  n LS  residuals 

s , • * 

*1» *n  • .=  : 

(b)  cMpute  the  ratio  of  the  nean  product  of  Buccesslee 
residuals  to  the  LS  variance  estlaator 


Once  this  estlnatlon  has  been  nade,  p uay  be  used  with  the 
Generalised  Least  Squares  aethod  to  reaove  the  autocorrelation  froa 
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I Since  p is  an  e8tliiiate»  and  not  an  exact  value  for  p»  it  la  unlikely 
' that  th^  process  will  renove  all  autocorrelation  on  the  first 
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iteration.  For  this  reason^  noltiple  iterations  are  generally 
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TABLE  X 

QUADRATIC  CURVE  FIT— LOWER  HALF 


NAME 

A 

(t) 

(t) 

F 

D.W. 

•dJR^ 

MSE 

A-10  SytCem 

-.0786 

.699 

< 2.96) 

.997 

( 3.09) 

800.58 

1.82 

.992 

.000059 

Englnt 

-.0102 

.0198 

(.0717) 

1.77 

( 5.23) 

419.36 

1.88 

.981 

.000X55 

Gun 

-a0125 

.712 

( 2.18) 

1.17 

( 2.84) 

537.93 

1.96 

.989 

.000065 

Milestones 

-a0384 

l.lll 

(15.49) 

.498 

( 4.31) 

3081.3 

.82 

.996 

.00025 

ACM  65  A 

• 0099 

.5365 

( 1.21) 

1.69 

( 2.31) 

80.89 

2.76 

.975 

•0029 

AUACS 

•0387 

.9532 

( 8.69) 

.733 

( 3.57) 

1013.10 

2.06 

.998 

.00015 

B-1  Model  1 Cost 

Estimates 

.7528 

4.829  ' 

(12.38) 

-3,498  (-8.19) 

392.19 

2.33 

.989 

.00021 

B-1  Model  2 Cost 
Estimates* 

218^729 

-4842 

(-8.03) 

31.901 

(15.68) 

1328.74 

1.90 

.998 

147.123 

F lOS  Milestones 

• 0534 

.3839 

( 1.05) 

3.528  ( 3.5) 

137.74 

2.62 

.986 

.000668 

F 100  Milestones 

•0258 

.5122 

( 2.41) 

.5411  ( .922) 

79.2 

2.16 

.963 

.00026 

C 141  Milestones 

•0496 

.2018 

( 1.55) 

.3189 

< 3.93) 

734.57 

2.08 

.997 

.00022 

C*5  Cost  Initial  Report 
Engineering 

-.0252 

1.514 

(19.67) 

5.594 

(14.79) 

13031.31 

1.92 

.999 

•000011 

C-B  Non-Recurring  Cost 

-.0342 

.1035 

( 9.12) 

.2478 

( 7.94) 

3698.91 

1.57 

.999 

.00002 

XB-70 

-.0306 

.3080 

( 2.05) 

1.732 

( 7.62) 

1031.12 

1.69 

.996 

.00021 

Tug  R 

.1417 

-1.523 

(-3.71) 

3.995 

( 6.46) 

108.1 

2.03 

.997 

•00094 

Tug  C 

.0200 

.3031 

( 5.81) 

1.582 

(17.62) 

3763.39 

2.64 

.999 

.000044 

Tug  E 

.0232 

.2418 

( 3.71) 

1.964 

(16.1) 

2714.81 

2.12 

.999 

.000057 

Tug  G 

• 0274 

.2717 

( 4.80) 

1.963 

(18.55) 

3769.95 

2.49 

.999 

.000044 

C-5  Quality  Assurance 
Hours 

• 1719 

-1.755 

(-9.19) 

5.04 

(16.3) 

848.45 

2.17 

.986 

.000024 

C-5  Production  Hours 

• 1443 

-1.686 

(-10.49) 

5.215 

(20.42) 

1576.09 

2.07 

.993 

.000023 

B-1  Engineering  Labor 
(Budget) 

-.0635 

1.39 

( 5.66) 

.6066 

( 1.22) 

550.97 

2.02 

.995 

.00024 

B-1  P & M Ubor  (Budget) 

-.0229 

1.39 

(15.63) 

-.128 

(-.632) 

1631.24 

2.13 

.998 

.000079 

B-1  Total  Labor  (Budget) 

-.194 

1.88 

( 4.17) 

.47 

( .587) 

490.26 

2.13 

.995 

.00022 

B-1  Manufacturing  Labor 
(Budget) 

-.075 

.449 

"(  .823) 

3.63 

( 3.35) 

215.29 

1.78 

.986 

..00099 

* Mot  Porcontago  Data 
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TABLE  III 

QUADRATIC  CURVE  FlT-*-UPPER  HALF 


<-’»• 


NAME 

A 

h 

(t) 

(t) 

f p. 

D.W. 

•am* 

MSE 

A-10  Sytcem 

-.83 

3.64 

(29.97) 

-1.76 

(-22.79) 

4312.53 

2.10 

.998 

.000016 

Engint 

-1.76 

S.86 

( 4.57) 

-3.10 

(-  3.95) 

32.19 

1.66 

.795 

.00028 

Gun 

-.66 

3.60 

(21.57) 

-1.95 

(-17.57) 

1102.23 

1.67 

.994 

.000066 

Mlltitontt 

-.0881 

1.94 

( 3.56) 

- .87 

(-  2.5) 

78.55 

1.92 

.881 

.00012 

ACM  6S  A 

-.33 

2.93 

C 5.16) 

-1.64 

(-  4.08) 

54.12 

2.88 

.964 

.00086 

AWACS 

-.3008 

3.306 

(14.46) 

-1.8 

(-11.98) 

423.37 

2.14 

•994 

•000082 

B-1  Modtl  I Cost 
Estimates 

.3804 

1.19 

(18.19) 

-.5794 

(-14.13) 

1335.68 

2.21 

.997 

.0000032 

B-1  Model  3 Cost 
Estimates 

-163.377 

31.46 

(5.84) 

-.1836 

(-  4.8) 

45.34 

1.93 

.936 

1159.55 

F-105  Milestones 

.2225 

1.1736 

(11.58) 

-.4080 

(-  5.69) 

1729.57 

2.09 

• 998 

.000084 

F«100  Mllesconss 

-.7607 

3.664 

(12.23) 

-1.939 

(-  9.07) 

403.02 

1.94 

• 987 

.00060 

C-141  Milestones 

-1.43 

4.825 

( 7.97) 

-2.397 

(-  6.29) 

257.4 

2.51 

.987 

.00028 

C-5  Cost  Initial  Report 
Engineering 

.4708 

1.133 

( 8.43) 

-.6069 

(-  6.44) 

128.76 

2.04 

.927 

.000021 

C-5  Non-Recurring  Cost 

.0387 

1.612 

(11.75) 

-.6741 

(-  7.49) 

1011.28 

1.77 

.994 

.000021 

XB-70 

-1.044 

4.365 

(11.21) 

-2.325 

(-  8.85) 

450.06 

1.63 

.992 

.00011 

Tug  R 

-2.051 

6.747 

(18.39) 

-3.711 

(-14.86) 

693.12 

1.92 

.996 

.00034 

Tug  C 

-.6194 

3.146 

(14.89) 

-1.521 

(-10.57) 

908.54 

2.36 

.997 

.00011 

Tug  E 

-.3129 

2.499 

(22.59) 

-1.185 

(-16.69) 

3104.38 

2.42 

.999 

.000014 

Tug  C 

-.4153 

2.912 

(12.04) 

-1.501 

(-  9.38) 

451.56 

2.17 

.993 

.000099 

C-5  Quality  Assurance 
Hours 

-.3756 

2.962 

( 2.86) 

-1.595 

(-  2.63) 

7.92 

1.80 

.323 

.000019 

C-5  Production  Hours 

.9660 

-.0948 

(-.248) 

.1208 

( .492) 

1.75 

1.89 

.057 

.000024 

B-1  Engineering  Labor 
(Budget) 

-.203 

2.49 

(184.4) 

-1.29 

(-136.09) 

102,647 

2.05 

.999 

.00000088 

B-1  P & M Ubor 
(Budget) 

-.613 

3.15 

(19.93) 

-1.54 

(-15.94) 

1384.46 

2.00 

.997 

.0000081 

B-1  Total  Labor 
(Budget) 

.149 

1.76  . 

(13.54) 

-.91 

(-10.63) 

519.66 

2.22 

.991 

.000027 

B-1  Manufacturing  Labor 
(Budget) 

.523 

1.06 

( 2.89) 

-.588 

(-  2.46) 

13.73 

s 

1.99 

.739 

.00016 

ikl^ 
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TABLE  VI 


INFLECTION  POIOT  LOCATIONS 


X Expenditures  > 


or 


X Hilestones 


E“1  * r 

F-103  Milestones 
C-141  Milestones  .*-^1 
C-5  Non-Recurring 

Tug  R 

Tug  c 

Tug  E 

Tug  C ^ ^ 

R-1  Engineering  Labor 
R-1  P b M Labor  ^ 

E-1  Total  Labor 
R-1  Manufacturing  Labor 
' C-5  Quality  Assurance  Hours 
C-3  Production  Hours 
A-10  System 
A-10  Engine 
A— iO  Cun 
A-10  Milestones 
AGM-63  A 

AHACS  - 


I .*  ■ 

'•esric. 

c V;*:' . 


.5385 
.5155 
.4659 
.5316 
.5026 
.6394 
.6626 
.3169 
.4681 
.4218 
".r'  .5866 
-i  - .4363 
*5246 
.4934 
.6899 
.5582 
.5388 
.6221 
■A  ..  .7011 


Mean  Inflection  Point  Locations: 
(The  four  B-1  labor  categories  are 
not  included  in  these  calculations 
to  avoid  duplication  of  data.] 


Standard  Error: 


.0852043  - y 


- .0729744  - lo^ 


.0053252 


- :*v.5. 

- ••‘jo'i; . 

• •r'fifitG, 

I . 

I'T'flAr; 


Z TiM 


-;r 


.562 


.3594 
.3400 
-.5050 
.3400 
.4431 
.5500  ’ 
.4600 
.5050  ■ 
.5050 
.3590 
.3590  . 
.3010 
.3590 
.4616 
.4769 
.4741 
.6245 
.4500 
.4205  , 
.4343 
.5050 


.462 


-I 

iti 

is,:  " 


r\V'  .• 


ri\: 


expenditures 
• • 

S «l/  .04668^ 
* P 16 


S - .0340189  - 
e 


■/ 


ZTT 

n— 1 


f 


.002918 


lo 
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TABLE  VII 


- . 1 • 

PROGRAM  B DATA  - 

PERCENTAGE  OF  LEVEL  6 PROPOSED  BUDGET  FIGURES 


Q/YEAR 

TOTAL  TIME 

2/1 

.0100 

. ^ '1;  ■ 

4/1 

.0680 

.2/2 

.1260  ■ 

4/2 

.1850  .c. 

2/3 

.2430  * 

4/3 

.3010 

2/4 

.3590 

4/4 

.4180 

.2/5 

.4760 

4/5 

.5340 

2/6 

.5920 

4/6 

.6510 

2/7 

.7090 

4/7 

.7670 

2/8 

.8250 

4/8 

.8840 

2/9 

.9420 

4/9 

1.000 

TOTAL  BUDGET 

ENC.LABOR 

.001206 

.001905 

.025086  • •• 

.045683 

.070831 

.m638 

.158547  - - 

.199698 

' .284803 

.3U899 

1 .421767 1 

.414406 

.557690  1 

1 .516945 

.668184 

.611508 

.759586 

.690111 

.833228 

.759773 

.874918 

.818965 

.908006 

.872539 

.937340 

..914294 

.960294 

.947851 

.978752 

.974348 

.991546 

.990384 

.993368 

.998021 

.999999 

.999999 

MFG.LABOR  P.M.LABOR 


.000015  .001322 

.001122  ' '*.058681 

.019189  .148909 

.101142  .229059 

.251838  .315084 

.407334  .390737 

f-. 

- l 

* -*u 

• -4 

. > -'i  ' 

Hi.  i-r; 

- a, 

.586607 

.468142 

Inflect 

Points 

.722934 

.533521 

.83054 

.597940 

.925338 

.661209 

‘fi 

— 

.955615 

.722603 

.965701 

.786322 

>1 

.974886 

.849762 

* 

.983690 

.902436 

.992814 

.940941 

.998511 

.970686 

.999866 

.991682 

.999866 

.999999 

I ' ^ 


KEY:  Percentages  in  blocks  are  inflection  points 


UNCERTAINTY  ASSESSMENT  FOR  THE 

bEvgl6f>HgHT  Contract 


#.  * ***rt^"»  Foifce  Institute  of  Technology 

Capt  Hlnlaa  L.  Glover,  Air  Force  Business  Research  Manageaent  Center 
Capt  John  0.  Lenz,  Air  Force  Systeas  CCmnand 


■ . IHTROOUCTION  ’ ■ ' * ' . Wr 


Cost  growth  has  been  a prevailing  characteristic  of  major  weapon 
system  development  programs  In  the  Department  of  Defense  (DOD)  for  the 
past  twenty  to  twenty-five  years.  As  a consequence,  a critical  prob- 
lem for  procurement  and  acquisition  manageaent  has  been  to  realistically 
forecast  program  cost  and  to  effectively  adrtnister  their  programs  and 
contracts  to  minimize  the  probability  of  a cost  growth. 

. . ^ planning  phases  for  a new  weapon  system  are  characterized 

^ of  technfcal  and  cost  unoartalnty  as  related  to  tiie 

development  and  production  of  a new  weapon  system.  During  these  early 
planning  stages,  cost  estimates  are  formulated  for  the  potential  Invest- 
meirt  necessary  to  acquire  and  maintain  a system.  As  knowledge  Increases 
and  better  Information  becomes  available.  Initial  cost  estimates  are 
ravised  and  updated  at  various  points  of  the  acquisition  life  cycle; 
Tnare  are  many  tools  available  to  managers  and  their  staffs  for  generat- 
ing cost  estimates  during  early  planning  and  development  phases  of  a 
weapon  system.  However,  very  few  of  these  methods  Include  a formal 
technique  for  evaluating  the  magnitude  of  the  uncertainty  associated 
^th  the  technfcal  aspects  of  a program.  These  considerations  form 
the  basis  of  several  research  efforts  which  are  the  basic  thae  of  this 
paper.  Each  research  effort  and  Its  results  will  be  described.  Then, 
the  Implications  of  these  results  will  be  presented. 


COMPLETED  RESEARCH 


The  series  of  research  efforts  conducted  at  the  Air  Force  Institute 
of  Technology  (AFTT)  evolved  from  an  entropic  cost  model  formulated 
under  Air  Force  sponsorship  at  the  University  of  Oklahoma  In  1971  (8). 
Before  discussing  the  research  studies,  a brief  description  of  the 
entropic  cost  model  Is  In  order. 
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The  Entropic  Cost  Model; 

To  facilitate  understanding  of  the  entropic  cost  node1»  a few  points 
■ust  be  Introduced  concerning  the  er.vlrorment  relative  to  the  develop- 
■ent  contract,  that  Is,  one  awarded  In  the  conceptual  or  validation  phase 
of  the  weapons  acquisition  process. 

Froa  a contractual  perspective,  there  are  two  key  tiae  periods  which 
relate  to  potential  cost  growth:  pre-contract  award  (pre-award)  and 
post-contract  award  (post-awa.'d) . (See  Figure  1.) 
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Figure  1 
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During  the  pre-award  time  frame  of  the  contract  life  cycle,  manage- 
aent  Is  primarily  concerned  with  strategy  fonaulatlon  to  deal  with  the 
developaent  of  a weapon  system.  Plans  are  made  to  transform  these 
strategies  Into  tactics  to.  If  possible,  preclude  cost  growth  during 
development.  To  accomplish  these  tasks,  management  has  at  Its  disposal 
certain  Information  that  should  permit  the  structuring  of  decisions  at 
the  time  of  award  of  the  per^nent  development  contract.  This  Informa- 
tion Includes  technical  Information,  cost  estimates,  and  the  results  of 
risk  analysis.  Techrrfcal  Information  Includes  engineering  estimates 
and  feasibility  studies  conducted  by  the  government  or  government  coir- 
tractors.  Cost  estimates  are  usually  available  froa  four  sources:  the 
Cost  Analysis  Improvement  Group  (CAIG);  Independent  Cost  Analysis  CICA); 
the  buying  organization  or  program  office  (PO);  and  finally,  defense 
contractors.  An  assessment  of  uncertainty  Is  critical,  however  rigorous 
and  formal  treatment  of  this  element  has  not  been  an  Integral  part  of 
management's  Information  system.  Only  recently  have  serious  efforts 
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besn  attempted  to  structure  the  process  of  analyzing  the  uncertainty 
which  confronts  managers  responsible  for  the  development  contract  and 
the  related  program.  , , u. 

1 

■hiring  the  post>award  time  frame  (see  Figure  1),  management  must 
monitor  its  information  system  and  take  actions  to  preclude  or  minimize 
cost  growth.  However,  the  inforaation  which  makes  up  the  control  base 
of  the  development  contract  has  its  roots  in  the  pre*award  time  frame. 

The  most  critical  point  in  time  for  model  effectiveness  from  a planning 
standpoint  is  Just  prior  to  contract  award  (see  Figure  1). 

The  infonaation  set  relative  to  the  develoiaent  contract  consists  of 
two  subsets:  an  ordered  set  and  a set  that  is  disordered  (lacks  order). 
The  ordered  set  is  composed  of  factors  which  appear  to  have  certain  out- 
comes during  development.  This  ordered  set  foms  the  basis  for  the  tar- 
get cost  (or  the  theoretically  "knst  likely”  cost)  for  the  developnent 
contract.  The  disordered  set  relates  to  factors  with  uncertain  outcomes 
and  forms  the  basis  for  cost  growth  during  the  post-award  time  period.  ' - ' 
This  information  set  is  illustrated  in  Figure  2.  U- 
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The  disorder,  or  uncertainty,  in  information  provides  the  foimdation 
for  the  entropic  cost  model.  In  set-theory  terminology,  disorder  is  the 
compleront  of  order  in  the  information  set.  For  the  entropic  cost  model, 
this  disorder  portion  of  the  information  set  is  approximately  equal  to 
entropy.  The  theory  of  entropy  derives  from  thermodynamics.  In  a 
physical  s^tem,  the  amount  of  disorder  is  a function  of  its  molecular 
state  at  different  teoperatures.  The  concept  of  entropy  was  extended 
to  information  systems  in  the  develoFmient  of  communication  and  information 
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theory  by  Shannon  and  Ueaver  when  they  attanpted  to  explain  noise  in  such 
systems  (9).  The  concept  is  further  extended  to  fom  the  basis  for  the  - I 
entropic  cost  model  which  is  formulated  as  follows:  i 


Final  Cost  « 


cr-’  «t.  :■  --i. 

' t : : -t'lx  l 3d  \ 'rotdim«r  ^ 
■'  |3>  S.-'-Vj  i 'wb;  ^0  ,, 

Target  Cost  ^ _ t^- w'a'iri  tirfT  ‘ 

1r  Disorder  in  Information  '^?KO}bf|id2 


Target  Cost 

Order  in  Infonaation 


'c  r Target  Cost  ?’v?  sKT^^- L -i 

1 • Entropy  ■ -•  b-jfot'fjyiri  i 

~,ito  sv^v'i'tu  -jMtJts  ifc<a«»  ,.>c if  ^ 

In  the  entropic  cost  model,  when  order  is  maximua  or  at  unity,  the  • ^ 
final  cost  is  certain  or  known  with  certainty.  However,  when  lack  of  ■ 

order  or  some  degree  of  disorder  is  present,  there  are  several  possible  ' 

outcomes  and  final  cost  is  not  certain.  Therefore,  uncertainty  relates 
to  multiple  outcomes.  The  purpose  of  the  entropic  cost  model  is  to 
estimate  final  cost  by  using  a quantitative  expression  of  the  disorder 
or  uncertainty  concerning  development. 


The  basic  goal  of  the  first  two  research  efforts  at  AFIT  was  to  test 
the  validity  of  the  model  using  simulated  data  from  development  contracts 
and  programs.  The  third  effort  was  conducted  to  determine  if  a best 
method  was  available  to  structure  and  generate  data  to  feed  the  model. 
The  following  paragraphs  provide  a description  and  the  results  of  each 
effort. 


; > * ; 

.V  > 

Glover  and  Lenz  ' . ' - '*  ; 


Research  Description.  Glover  and  Lenr  gathered  data  from  the  Short 
Range  Attack  Missile  (SRAM)  development  program  for  use  in  testing  the 
model  (4).  The  researchers  reviewed  contract  files  and  program  documen- 
tation to  determine  the  data  availability  for  the  test.  During  this 
. review,  it  was  found  that  the  data  (element  weights)  necessary  to  test 
the  model  was  contained  in  source-selection  files.  However,  these  files 
were  not  readily  available  because  of  their  sensitivity.  Validation 
required  the  creation  of  the  information  set  in  existence  imoediately 
after  the  award  of  the  SRAM  development  contract.  Limited  access  to 
source  selection  records  required  the  researchers  to  develop  a method 
to  simulate  and  quantify  the  uncertainty  of  information:  The  method 
had  to  meet  the  requirements  of  research  and  be  potentially  useful  to 
managenent  to  capture  and  measure  uncertainty.  _ 

'■}'  >' 

The  work  of  C.  Jackson  Grayson  in  his  study  of  uncertainty  and 
risk  and  its  application  to  oil  and  gas  drilling  decisions  was  known  to 
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the  researchers  (6).  Jackson's  technique  relied  on  the  use  of  probabfl* 
Ity  statements  by  experts  in  oil  and  gas  exploration  and  drilling  relative 
to  success  in  terms  of  production.  Grayson  cor.jidered  both  individual 
and  group  risk  preferences  in  his  moHc.  A similar  approach  based  on 
subjective  probabilities  mbs  perceived  to  be  applicable  to  the  develop* 
■ent  contracting  environment  in  the  ODD. 

Consequently.  Glover  and  Lent  conducted  an  exhaustive  review  of 
the  techniques  available  for  measuring  subjective  opinions  of  experts. 

The  review  identified  the  DELPHI  method,  developed  at  the  RAND  Corpora- 
tion. as  the  best  approach  to  be  used  to  simulate  the  development  contract 
award  environment  for  the  SRAH.  Generally,  the  DELPHI  technique  is  used 
to  solicit  expert  judgment  to  forecast  what  might  happen  in  the  future. 
Glover  and  Lenz  used  the  structure  and  method  of  DELPHI  to  simulate  what 
happened.  They  used  a series  of  controlled  Interviews  with  feedback 
directed  at  individuals  who  actually  participated  in  SRAM  source  selec- 
tion. The  researchers  solicited  statements  of  probability  concerning 
outcome  measures  of  unacceptable,  acceptable,  and  exceptional  for  each 
of  nine  (9)  factors  of  concern,  such  as  range,  speed,  etc.  ^ / 

Using  this  approach.  Glover  and  Lenz  were  able  to  identify  some 
19.683  possible  factor  - outcome  coci>inations  and  a probability  for  each. 
The  calculation  base  was  the  three  categories  of  outcome  identified  dur- 
ing the  simulation  for  each  of  the  possible  factors.  Therefore,  there 
were  y or  19.663  possible  outcomes  for  the  development  program. 

Entropy  was  then  calculated  as  follows:  ''  ' 

' 19,683 

..  . t p,  109P, 

i ■ 1 

log  19.683 


- 0.686 


Where:  0 <.  Entropy  < 1 . and 

p^  ■ probability  of  the  i^  factor  outcome  (2). 


Using  the  entropy  value  of  0.686.  an  estimate  for  total  program 
cost  as  follows: 
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Total  Progran  Cost 


/ i 


, Target  Cost 
I - Entropy 

. ..  1 - 0.686 

1 Ji43.3H 
OTT- 

- $456.4H 


■ r ^ 
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^ develop«nt  progra 


Research  Ftndfr 

was  TO  mil  Hon.  ,„r  , 

entropic  cost  oodel  mss  $456  Bill  Ion  TM« 
t«riii9  th.  wrst  possible  cost  aSlMoS  3oriS^?SiS^  tII.21‘2!::: 

.f.»i?Kvri;.s:!?orini^?nT-»s;pr^^ 

^ r^earch  effort  iiere  not  considered  adequate  to  validate  the  md^l 
Therefore,  another  teaa  of  researchers  at  AFIT  Bahi;ir7  nn/i 
atteiqpted  to  replicate  the  Glover  and  Lenz  approach  (1). 


Bablarz  and  G1edras> 


PescHptlono  Bablarz  and  Giedras  continued  the  effort  to 
test  the  entropic  cost  model  by  ropllcatfng  Glover  and  Leny*« 

£“pa7so':;sr‘^^u^:'=^^,»„^To^^ 

Slrtros  obtalpod  molts  that  «.«  d1ffomrf«,IX”r’o?S}|^  “Jd 

Research  Rndings.  Bablarz  and  Giedras  found' that  the  apolicatior 
of  an  approach  such  as  OaPHI  did  not  ptx)vide  evidence  to  substaSJil?!’ 

5J^“«  0'  if*  « • forocaSlXI  Sol!  {taS  o“fort 

P^uced  a final  result  that  was  different  from  the  Glover  and  Lenz's 
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Using  the  results  of  their  uncertainty  assessment.  Babiarz  and 
Giedras  obtained  subjective  probabilities  for  only  the  "Mor"  (or 
unacceptable)  and  "acceptable**  categories  for  twelve  (12)  factors  of 
concern  at  the  time  of  F-SE  development  contract  award.  Therefore, 
there  were  2^2  or  4096  possible  outcomes.  Entropy  was  then  calculated 
as  follows  I t * z >i  2- * tji  1- 


-j-  • • 


?•  I—  g ■ j 


Entropy  ■ 


r.  : 


Sys^  Entropy 

Naximui  Entropy 


A >/>  t*??  i 9n}t  tn. 

■ ' - ?5o  4-  i"! 


:.fk  'f  ■ c:!t  ir'  • « 

'•*  4096  * - - 

;*  •>>  vi  p log  p. 

•'i  .nj  si?. ' 

•. -t-t*  ' * * * 

log  4096 


~ hot  ^avofil'  • 


■ft';- 


'oW;-r;.C 


0.910 
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Using  this  result,  total  program  costs  were  then  calculated  as  follows: 

Total  Pn,5«.c5t  - ?rag^  ' ' ;; 

* _ $83.63^ 

. g ' • rnrsTO  ^ - 

_ $83.635M 
T.  OS  ■ 

- $929.2781 


- The  actual  cost  for  the  development  of  the  F-5E  was  $89,322  mil- 

lion. Thus,  the  model  forecast  of  total  program  cost  was  10.4  times 
greater  than  the  actual  cost  (1.  81).  Based  on  this  result,  the 
researchers  questioned  the  use  of  a device  such  as  DELPHI  to  generate 
'measurements  of  uncertainty.  The  problems  encountered  did  not  relate 
to  the  model,  but  measurement  of  uncertainty.  Although  the  basic 
' _ approaches  used  by  the  researchers  on  both  efforts  were  the  same,  there 
were  also  some  subtle  differences  which  need  to  be  discussed  before 
describing  the  last  research  effort  which  specifically  addresses  mea- 
( ; • suring  uncertainty  (5). 
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Lanclos  critiqued  the  application  of  DELPHI  to  simulate  uncertainty 
at  the  time  of  contract  award  (7).  Lanclos  treated  the  two  efforts  as 
case  studies  and  assessed  the  differences  between  them.  The  following 
is  a discussion  of  Lanclos*  study  findings  (7^  9>11). 

Glover  and  Lenz  selected  four  persons  as  their  expert  panel  who 
were  actually  members  of  the  SRAM  So'irce  Selection  Evaluation  Board 
(SSEB).  Further,  Glover  and  Lenz  followed  up  their  research  effort 
by  gaining  access  to  SRAM  SSEB  files  and  were  able  to  verify  their 
•aicertalnty  assessments  In  all  but  one  area  of  concern.  Therefore, 
ute  team  was  strongly  confident  In  the  assessment. 

Bablarz  and  Gledras  decided  to  use  the  DELPHI  approach  from  the 
start  of  their  effort  and  did  not  attempt  to  review  SSEB  files  for  the 
F-5E.  Their  panel  members  were  not  members  of  the  F«5E  SSEB,  but  had 
managerial  positions  In  the  F-SE'  program  office  during  the  development 
of  the  F-SE,  one  of  which  had  to  drop  out  of  the 'assessment  due  to  a 
temporary  duty  assignment.  Bablarz  and  Gledras  also  asked  for  concerns 
during  development,  not  at  contract  award.  Some  of  the  areas  of  concern 
Identified  occurred  as  late  as  two  years  Into  the  F-SE  development  pro- 
gram. 

Having  studied  the  significant  differences  between  the  first  two 
efforts  to  test  the  entropic  cost  model,  Lanclos  concluded  that  both 
efforts  would  have  been  more  realistic  If  they  had  been  conducted 
"real-time.'*  Also,  Bablarz  and  Gledras*  choice  of  experts  and  prob- 
lem areas  did  not  give  a true  representation  of  the  uncertainty  In 
F-SE  development  at  the  time  of  contract  award.  Finally,  the  validity 
of  DELPHI  as  an  approach  to  measuring  uncertainty  was  not  proved  or 
disproved  by  either  effort.  However,  Its  use  to  structure  the  assess- 
ment of  uncertainty  does  have  merit  (7,  11-12). 

The  concern  of  measuring  uncertainty  was  addressed  by  a third 
research  effort  at  AFIT  In  1976  (S). 
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Grayson  and  Lanclos 


Research  Description.  Grayson  and  Lanclos*  research  objective  was 
to  evaluate  existing  techniques  for  assessing  subjective  probability 
(5,  9-10).  Their  objective  was  to  propose  an  approach  to  assess  and 
measure  the  uncertainty  which  existed  relative  to  a given  weapon  system 
development.  Their  study  addressed  the  following  research  question; 
"Uhat  existing  subjective  probability  assessment  technique  would  best 
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assess  the  magnftude  of .uncertainty  In  a given  weapon  systea's  develop- 
Bent  effort"?  Grayson  and  Lanclos  conducted  an  In-depth  review  of  the 
critical  literature  available  concerning  techniques  and  nethods  for 
assessing  risk  and  measuring  uncertainty  concerning  future  events.  In 
their  study,  the  researchers  considered  the  statistical  and  psychological 
aspects  of  subjective  probabilities  and  applied  content  analysis  to  six 
techniques  for  assessing  subjective  probabilities  which  have  been  used 
In  the  past.  These  techniques  are 


i,.' 


Cholce-BetMeen-Ganbl es 
Standard  Lottery 


V ^ . r 


Nodi fled  Churchman-Adcoff  . 
DELPHI  • . * - . 4 


t(5)  > DeGroot  Consensus 
. ' (6)  Direct  Estinatlon 
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The  study  Included  a description  of  each  technique  and  an  application  to 
an  example.  Each  technique  will  be  briefly  described  for  reference  pur- 


poses. 


' The  Choi ce-Be^en-Gairi)1  es  Is  a technique  that  enploys  betting- 
type  situations  relative  to  gambling  In  an  attempt  to  obtain  probability 
and  cunulative  density  functions.  To  accomplish  this,  experts  are 
offered  choices  between  a "real-world"  gambling  situation  which  Includes 
values  for  an  Item  with  unspecified  probabilities  and  a hypothetical 
situation  about  two  outcomes  with  probabilities  specified.  A'  series 
of  Iterations  that  Include  varying  probabilities  are  conducted  to  derive 
the  density  functions  desired. 


The  Standard  Lottery  Is  a technique  for  deriving  a probability 
density  function  whicn  coven  all  possible  values  of  a given  Item  charac- 
teristic. It  Is  similar  to  the  Cholce-Between-Ganbles  In  that  It  Is. 
used  to  present  an  expert  with  two  gambling  situations.  But  it  also 
differs  from  the  Cholce-Between-Ganbles  as  it  does  not  Involve  varying 
probabilities  during  the  Iteration  process  until  a point  of  Indifference 
Is  reached.  The  technique  Involves  selecting  a nunber  of  lottery  tickets 
from  a pool  of  100  that  is  varied  to  achieve  the  point  of  indifference. 
The  expert  can  chose  as  many  as  he  wants.  After  the  choice  Is  made,  a 
ticket  Is  drawn  at  random.  The  lottery  Is  used  as  a standard  comparison 
to  assist  the  expert  to  decide  on  a probability  value  for  a certain  out- 
come which  relates  to  a characteristic  of  performance  for  an  Item. 


The  Churchman-Ackoff  technique  does  not  use  a gambling- type 
situation  or  a level  of  Indifference  concerning  characteristics  or 


outcomes.  The  expert  is  confronted  with  evaluations  using  judgments 
that  require  ranking  of  relative  probabilities  between  sets  of  values. 
For  example,  greater  than,"  "equal  to,"  or  "less  than"  choices  are 
used.^  The  expert  must  decide  ranges  of  possible  values  for  "real- 

function^^^3o"L)^***  converted  to  a probability  density 

The  DELPHI  technique  is  used  to  solicit  group  consensus  about 
possible  outcomes  or  events  in  the  future.  Members  of  the  group  of 
experts  not  privy  to  the  other  participants'  identities.  This 
feature  minimizes  the  possibility  of  dominant  personalities  and  group 
prnsure  in  obtaining  final  results.  The  method  involves  rounds  of 
intemgation  which  include  controlled  feedtock  between  rounds.  Each 
participant  can  change  his  previous  choice  based  on  the  feedback  from 
each  round.  These  Iterations  continue  until  no  change  occurs.  The 
results  are  then  used  to  assess  and  form  a measure  of  group  consensus. 


The  DeGroot  Consensus  technique  also  uses  experts  to  reach  a 
**"**"*“*• However,  each  member  is  required  to  evaluate  a proba- 
bility for  an  unknown  value  of  a parameter.  Then  each  participant  is 
exposed  to  the  inputs  from  other  participants  and  revises  his  own  judg- 
ment after  assessing  the  others'  importance,  expertise,  etc.  This 
process  continues  until  no  more  changes  occur  (5,  39), 


..  Direct  Estimation  involves  an  estimating  effort  that  gives  ft 
1U  name;  each  expert  approximates  or  estioates  probabilities  directly 
with  no  e;;posure  to  other  probabilities  or  situations.  These  results 
are  used  directly  to  derive  a probabilities  distribution  for  future 
events  or  outcomes  (5,  40). 

Research  Findings.  Grayson  and  Lanclos  applied  content  analysis  to 
**  j ^®^hnique  studied  to  determine  which  technique(s)  was(were}  commonly 
used  for  measuring  uncertainty.  The  researchers  found  that  DELPHI  has 
been  widely  critiqued  in  the  literature,  and  its  use  is  widespread  and 
popular. 


The  team  also  found  that  critical  analysis  and  testing  of  the 
six  techniques  were  fragmented  and  lacked  continuity  and  consistency. 
Based  on  these  findings,  Grayson  and  Lanclos  recoimended  that  a series 
of  tests  be  conducted  to  determine  which  techniques  or  combinations 
thereof  apply  to  measuring  uncertainty  for  weapon  systems  development. 
They  suggested  that  the  entropic  cost  model  could  be  used  to  measure 
and  compare  the  results  of  these  tests. 


Overall  Conclusions  from  Completed  Research 


Obviously,  validity  testing  of  the  entropic  cost  model  is  not  complete. 
The  Grayson  and  Lanclos  effort  supports  DELPHI  as  a popular  and  widespread 
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tool  for  measuring  uncertainty.  But,  popularity  and  widespread  use  does 
not  Indicate  that  a technique  is  the  "one  best  way."  However,  the 
research  does  provide  some  Implications  for  procurement  and  acquisition 
researchers  and  managers.  These  1^>11cat1ons  provide  a logical  baseline 
for  early  Identification  of  uncertainties  concerning  outcomes  for  weapon 
system  development.  Once  uncertainty  Is  known  to  exist  concerning 
development,  management  can  better  use  Its  Information  base  to  deal  with 
risks  related  to  the  uncertainty.  Development  of  such  a capability 
presents  a challenge. to  our  researchers  and  managers.  , ^ . u,] 
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Although  tntenstve  and  widespread,  research  has  not  progressed  to  the 
point  of  providing  wanaga^nt  with  an  effective  set  of  tools  for  realis- 
tically forecasting  the  ultimate  cost  of  a new  system.  Costing  method- 
ologies vary  and  few  emphasize  the  formal  use  of  methods  for  capturing 
and  measuring  technical  uncertainty.  The  problem  of  quantifying  uncer-  , 
tainty  requires  further  research  and  testing.  The  research  challenge 
is  to  Invent,  design,  and  innovate  methods  for  quantifying  uncertainty 
to  feed  madels  such  as  the  entropic  cost  model.  These  efforts  will  be 
difficult  at  best;  there  will  be  failures  and  hopefully  successes. 

However,  this  series  of  research  efforts  have  implications  fbr  management 
also.  The  qualitative  aspects  of  the  research  may  be  useful  fbr  stru^r- 
ing  the  assessment  evaluations  of  doubt,  reviewing  status,  and  improving 
planning  baselines.  Before  discussing  these  1iq>1i cations,  let  s first 
address  the  research  challenge. 


Wesearch  Implications  awi  Challenge 


Procurement  and  acquisition  processes  have  bean  tte  topics  of  a 
sizeable  portion  of  business  literature.  A good  portion  of  the  literal 
ture  that  criticizes  these  processes  Is  accompanied  by  suggested  alter- 
nate approaches.  But,  those  authors  which  criticize  for  that  purpose 
alone  must  be  questioned  as  to  motive.  However,  some  criticisms  are 
Justified.  Business  environments  are  not  discrete  like  those  o.  tha 
physical  sciences  and  are  much  more  difficult  to  comprehend  and  measure. 
Since  the  future  Is  difficult  to  forecast.  It  Is  certain  that  cost 
growth,  as  related  to  new  and  untried  program  concepts,  and  technology 
have  a high  probability  of  occurrence. 


Models  which  are  now  used  for  estimating  development  and  production 
costs  usually  apply  historical  data  to  forecast  the  future.  In  most 
cases  historical  data  does  exist  and  statistical  tools  are  available 
to  transform  the  data  Into  meaningful  Information  about  what  one  can 
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expect— a ssuntng  that  the  future  will  behave  similar  to  the  past.  Such 
oethods  are  fairly  well  tested  and  understood  by  our'cost  analysts, 
price  analysts,  contract  managers,  and  program  managers.  However,  some  - 
of  our  analysts  and  managers  become  so  engrossed  in  their  tasks  that  they 
are  vulnerable  to  many  of  the  pitfalls  of  their  Jobs  and  enviroraent  with- 
out realizing  their  predicament  until  it  becomes  a real  problem.  Many  of 
these  problems  have  their  roots  early  in  the  life  cycle  of  a system.  Our 
processes  demand  that  variables  and  factors  have  measurements  which  can 
be  translated  into  dollars.  If  a data  base  (as  we  know  it)  from  the  past 
is  available,  we  have  the  tools  for  forecasting.  But,  the  underlying 
implication  of  the  series  of  research  efforts  discussed  earlier  in  this 
paper  concerns  a different  data  base. 

The  contention  of  the  researchers  is  that  experts*  knowledge  and 
experience  are,  or  are  part  of,  a data  base.  Researchers  have  a unique 
challenge  to  develop  and  design  methods  and  techniques  to  capture  that 
knowledge  and  experience  and  docunent  the  results  for  application  to 
new  developments.  iHe  need  to  design  data  bases  which  can  be  used  by 
management  to  assess  uncertainty  and  to  estimate  program  costs.  First, 
a qualitative  baseline  of  factors  shoald  be  addressed—some  researchers 
are  uneasy  with  elements  of  a problem  or  study  that  cannot  be  readily 
quantified.  The  history  of  mathematics  reveals  many  such  problems. 

Theory  is  often  difficult  to  quantify.  However,  there  are  also  break- 
throughs recorded  in  the  study  of  our  so  called  purest  of  sciences. 

These  breakthroughs  did  not  occur  overnight— but  some  were  accidents.  • 


To  meet  this  data  base  challenge,  designers  should  at  first  consider 
the  qualitative  aspects  of  uncertainty.  The  first  step  should  be  to 
qualify  what  is  uncertain  and  then  apply  a logicially  structured 
approach  to  measure  the  uncertainty.  A potential  tool  is  the  entropic 
cost  model.  However,  methods  should  be  based  on  ■siin)licity.  Figures 


. 3a  and  3b  illustrate  a conceptualization  of  the  challenge 


Figure  3a 

Research  Challenge 


Minimize  the  Division 


The  Challenge 
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» Figure  3b 

wV-to  T ru  Research  Objective 


— a.  I 

. ' *T^  d ! . 


• : . ^'CLX^  ' ■ 

‘:ZB 

'T-  t 

ytf 

7 vx* ’'c-.r  j t,  ^ Objective  - fbtch  the  qualitative  aspects  idth  ^ 

. a quantitative  aeasure  for  uncertainty.  - 

^'Xy  .v  ■■^ ^^  .' 1 c . • -‘c  . J 1. 

# • 

Therefore,  Grayson  and  Lanclos  were  on  the  right  track  : addressing 
what's  available  for  measuring  uncet*ta1nty.  The  series  of  tests  would 
have  continued  possibly  on  a hlt-and-oriss  basis.  Each  approach  studied 
by  Grajrson  and  Lanclos  depends  on  an  Initial  qualitative  exercise  tc 
Identify  factors  or  aspects  of  the  future  which  are  uncertain  to  the 
evaluators  and  experts  that  are  familiar  with  the  development  program. 
Researchers  should  also  be  ready  to  "translate*  their  efforts  to  "real- 
world"  possibilities.  This  "real-world"  Is  where  our  operators  and 
managers  live  and  work — they  should  be  the  benefactors  of  research  and 
need  to  be  supplied  with  tools  that  work.  The  series  of  research  studies 
also  have  a "message"  for  nanagement  based  on  the  qualitative  nature  of 
the  techniques. studied  by  Grayson  and  Lanclos. 
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Management  Considerations 


Each  of  the  techniques  for  assessing  and  measuring  uncertainty  which 
were  studied  by  Grayson  and  Lanclos  depends  on  identification  of  areas 
or  factors  that  have  uncertain  outcomes.  These  areas  should  be  solicited 
from  program  participants  early  in  the  planning  cycle  prior  to  contract 
award.  Procurement  and  acquisition  tools  already  exist  for  documenting 
concerns.  Current  directives  specifically  address  minimizing  uncertainty 
and  risk  (3,  6)  during  the  acquisition  cycle.  Uncertain  factors  should 
be  identified  early.  Such  an  exercise  provides  a structured  approach  to 
capturing  the  judgment  of  managers,  evaluators,  and  experts;  key  parfci- 
pants  usually  have  a knowledge  and  experience  base  that  they  draw  on  JH 
"judge"  and  in  turn  identify  concern  or  uncertainty  based  on  available 


Infonoatlon.  Management  could  do  well  to  capitalize  on  this  knowledge 
and  experience  by  soliciting  participant  concerns  and  uncertainty  early 
In  the  pre-award  planning.  This  effort  should  be  a coEbfned  exercise 
by  both  the  contracting  officer  and  program  manager  supported  by  their 
fmctlonal  staff.  One  major  criticism  of  such  exercises  is  that  of 
tlne—but  some  time  spent  early  may  have  some  long  range  pajoffs  In 
terms  of  readiness  to  respond  to  contingencies  during  development  and  ^ 
to  minimize  downstream  costs.  • • I ' 

* Once  this  "uncertainty  baseline*  Is  established.  It  could  be  updated 
at  major  milestones  during  performance  of  the  development  contract.  . Such 
M approach  could  also  benefit  the  staff  In  preparing  fbr  major  system 
mlestones  rraulred  by  addressing  such  factors  In  the  Decision  Coordinat- 
ing Paper  (4). 

The  baseline  should  be  updated  Immediately  after  development  contract 
award  to  capture  specific  Information  relevant  to  the  particular  con-> 
tractor! s)  Involved  and  evaluator's  beliefs  In  their  capability  to  oei^ 
form  the  contract. 

The  thrust  of  the  Implications  are  not  really  new.  But  the  main 
purpose  of  this  discussion  Is  to  prompt  management  to  use  their  talent 
and  ability  to  establish  such  a baseline  for  dealing  with  uncertainty,  l 
The  ultimate  benefits  should  be  obvious:  better  visibility  and  under- 
standing of  uncertainty  assessment  problems  which  are  encountered  by 
contract  and  program  managers.  A structured  "audit  trail"  should  help* 
managers  to  get  at  the  "root  cause*  of  sane  of  our  problems  and  traat 
then  accordingly. 

If  a problem  Is  Identified  which  cannot  be  treated  readily,  at  least 
the  researcher  can  shed  some  light  on  an  approach  to  help  better  address 
the  problems  and  Issues  at  hand.  This  approach  takes  tine,  but  the 
potential  payoffs  may  greatly  exceed  the  Investment. 
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The  actual  text  of  LTC  Boyett's  talk  was  not  available  for  pii>11cat1on.  The  following  fs» 
howevert  a shortened  account  of  that  talk.  » . - 

^ '•  *«  * *1 V ^ I . -x> 


Defense  Oepartnent  profit  policy  Is  Interpreted  both  at  the  develogKnt  of  the  profit 
objective  point  In  the  procurement  process  and  at  the  conclusion  and  docuKnUtlon  of  the 
negotiated  results  point.  This  raises  the  questioo,  therefore,  of  how  000  buying  activities 
have  been  Iwplemiting  000  profit  policy-  To  Investigate  this  li^ortant  policy  paraoeter, 
a research  effort  was  developed  to  provide  an  answer  to  the  question:  Mhat  are  the  rela- 
tive contributions  of  cost  and  non-cost  profit  factors  of  the  Individual  procurement  action 
to  the  000  profit  objective  and  the  negotiated  profit  reported  on  Department  of  Defense 
(OOP)  negotiated  procurements? 


To  Investigate  the  research  question,  a daU  base  was  assembled  from  Information 
collected  by  the  AssIsUnt  Secretary  of  Defense  (Comptroller).  Basic  Information  was  Uken 
from  the  Defense  Department's  report  of  Individual  contractor  profit  plan,  DO  1499,  which 
Is  prepared  on  every  contract  over  $200,000  by  the  procuring  contracting  officer  at  the 
tine  negotiations  are  conpleted. 


To  reduce  computational  requirements  and  Insure  a relevant  data  base,  the  original  data 
base  which  covered  fiscal  years  1959  through  1975  was  reduced  to  the  top  50  contractors  by 
dollars  awarded  for  fiscal  years  1973,  1974,  and  1975.  To  Insure  that  the  data  base 
would  represent  the  najorl^  of  situations  In  which  000  profit  policy  ws  applied,  tests 
were  run  on  the  number  of  docaoents,  as  well  as  the  nuad>er  of  dollars  awarded  to  contractors. 


In  summary,  the  analysis  found  the  following  to  exist  with  regard  to  the  cost  and  non- 
cost  parameters  provided  by  Department  of  Defense  policy  for  use  in  establishing  contract 
profit  objectives: 

1.  Cost  Input  to  total  performance,  CITP,  Is  the  largest  single  factor  contributor  to 
the  DOO  profit  objective. 

2.  Cost  risk,  the  second  most  frequently  used  factor  Is  also  the  second  largest 
contributor  to  DOO  profit  objectives  (Tables  6 and  7). 

3.  Among  the  most  frequently  used  contract  types,  FFP,  FPI,  and  CPIF,  cost  risk  has 
been  widely  employed  In  addition  to  cost  Input  to  total  performance  (Table  6). 

4.  The  statistical  model  explains  771  of  the  variance  In  negotiated  profit  percentages 
using  only  the  factors  prescribed  by  DOO  profit  policy. 
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S.  Adding  the  contract  type,  contractor,  and  purchasing  organization  to  the  statistical 
■odel  Increased  the  Boders  explanatory  power  by  less  than  4S  collectively. 


6.  .Increases  In  the  CITP  factor  tended  to  be  offset  by  Increases  in  the  other  factors. 


7.  Decisions  to  employ  or  not  employ  a given  profit  factor  do  not  appear  to  Influence 
negotiated  profit  levels. 


8.  Use  of  profit  factors  varies  between  contract  types. 


» t . . 
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9.  The  selected  factors  and  special  profit  factors  of  000  profit  policy  are  used 
Infrequently,  and  when  used,  account  for  only  slight  anounts  for  the  000  as  a whole. 


10.  There  Is  no  evidence  to  suggest  that  the  cost  risk,  past  performce.  selected 
factors,  or  special  profit  factors,  when  used,  significantly  Influence  either  the  magnitude 
, of  000*s  final  negotiated  profits  with  Industry  or  the  allocation  of  profits  asong  contrac- 
tors. 
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For  the  Oepartment  of  Oefense  as  a whole  the  findings  Indicate  a cost-based  policy  / 
tempered  slightly  by  Judgmental  applications  of  risk,  past  performance,  etc.  This 
means  that  cost  and  Its  definition  Is  the  most  significant  single  Influence  on  profit 
objectives  and  outcomes.  . * - . 
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* A Major  finding  of  this  study  Indicates  that  idien  they  are  used  the  five  profit  factors 
Influence  negotiated  profit.  Cost  liput  to  total  performance  Is  by  all  ti:sts  the  dominant 
Influence.  This  finding  1^>11es  that  attempts  to  Influence  negotiated  profit  must  be 
coupled  with  policies  relating  the  use  of  profit  factors  which  are  precisely  worded  and 
loglcaUy  constrained  within  desired  limits. 
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Thi^ghout  000  and  the  Federal  Govenwnt,  there  Is  little  forwl  guidance 
. available  to  assist  contract  administrators  In  the  administration  of  the  fiscal 
*?**  **'“*  (CPFF)  procurements.  And  what 

little  DOO-wide  formal  guidance  there  Is  docs  not  provide  too  useful  a basis  of 
ue  resolution  of  problems  that  Invariably  arise.  At  the  Air  Force  Flight  Test 
center,  I have  developed  a si^>le  method  of  documenting  and  providing  for  the 
resolution  of  costs  associated  with  ovemms,  constructive  changes,  additions, 
and  ^letlons  to  the  statement  of  work  of  CR  and  CPFF  contracts.  The  method 
applies  to  forward  and  retroactively  priced  change  orders,  and  suoplemenUl 
agreements.  Before  the  ioploaenUtlon  this  method,  each  contractor  had  a 
different  method  of  proposing  costs.  This  lead  to  a great  deal  of  confusion  on 
the  part  of  both  contractors  and  Government  actarinlstratlon  personnel.  Casualties 
OT  this  process  Included:  Inadequate  P.L.  87-653  (Truth  In  Negotiations  Act) 
docimenution;  lack  of  clear  understanding  by  both  parties  of  the  real  problem 
and  the  associated  impact  on  cost;  unwillingness  to  address  the  real  Issues: 
etc.  I believe  that  cost-pi us-a-percentage-of-cost  (CPPC)  situations  arose, 
were  proposed  and  negotiated  due  entirely  to  an  unintentional  leek  of  under- 
standing and  disclosure.  1 have  also  found  that  the  00  Form  633-5  which  Is 
the  primary  vehicle  provided  adninistrators  in  contract  changes  does  not  provide 
a method  to  clearly  address  problems  or  to  Insure  that  any  previously  unidentified 
problems  are  resolved  before  changes  are  negotiated  to  CR  and  CPFF  completion 
5®**Tracts.  Indeed,  I consider  the  633-5  method  as  presently  shown  by  that  form 
to  be  useful  onl^  In  FFP  contract  changes.  It  has  been  tiy  experience  that  the 
proposed  method  identifies  problems,  and  provides  a simple  avenue  for  resolving 
corplex  contract  modification  situations. 

1.  THE  BACKGROUND:  Pertinent  to  the  subject  Is  a discussion  of  the  underlying 
factors  influencing  administration  of  CPFF  completion  contracts  a>  administration 
relates  to  the  resolution  of  fiscal  problems. 

a.  The  Armed  Services  Procurement  Regulation  sets  the  tone  of  CPFF  pro- 
curement in  3-405.6  as  follows: 

The  cost-plus-fixed-fce  contract  Is  a cost  reimbursement 
type  of  contract  wliich  provides  for  the  payment  of  a 
fixed  fee  to  the  contractor.  The  fixed  fee  once  nego- 
tiated does  not  vary  with  actual  cost,  but  may  be  adjusted 
as  a result  of  any  subsequent  changes  in  the  work  or 
services  to  be  performed  under  the  contract... 

The  Completion  form  Is  one  which  describes  the  scope 
of  work  to  be  done  as  a dearly  defined  task  or  job 
with  a definite  goal  or  target  expressed  and  with  a 
specific  end-product  required.  This  form  of  contract 
normally  requires  the  contractor  to  complete  and  deliver 
the  specified  end-product  (In  certain  instances,  a final 
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report  of  research  accoopllshlng  the  goal  or  target) 
as  a condition  for  payment  of  the  entire  f1xed>fee 
established  for  the  i<ork  and  irithin  the  estimated  cost 
If  possible;  however.  In  the  event  the  work  cannot  be 
completed  within  the  estimated  cost,  the  Govemraent  can 
elect  to  require  more  work  and  effort  from  the  contractor 
without  Increase  In  fee  provided  It  Increases  the  estimated 
. cost. 

CT  type  ^tracts,  and  more  particularly  CPFF  contracts  are  frequently  termed 
best  efforts  contracts.  This  term  Is  generally  taken  to  mean  that  a con- 
tractor Is  expected  to  diligently  pu<^ue  completion  of  the  required  effort 
within  the  funds  allocated.  However,  If  during  performance  It  appears  that 
funding  beyond  that  originally  contemplated  will  be  required,  the  contractor 
IS  not  obliged  to  continue  performance  beyond  the  face  value  of  funds  provided. 


b.  Thrw  clauses  are  the  primary  vehicles  In  the  administration  frame- 
Hit  CHANGES  clauses  for  cost  reimbursement  use,  primarily 

ASPR  7-203.2  and  7-404.1.  These  two  clauses  are  essentially  the  s^  for  our 
purposes.  The  applicable  portions  are  as  follows: 


If  any  such  change  causes  an  Increase  or  decrease  In 
the  estimated  cost  of,  or  the  time  required  for,  the 
performance  of  any  part  of  the  work  under  this  contract, 
whether  changed  or  not  changed  by  any  such  order,  or 
otherwise  affects  any  other  provision  of  this  contract, 
an  equitable  adjustment  shall  be  made: 

(1)  In  the  estimated  cost  or  delivery  schedule, 
or  both; 

(11)  In  the  amount  of  any  fixed  fee  to  be  paid 
to  the  Contractor;  and 

(111)  In  such  other  provisions  of  the  contract  as 
may  be  so  affected,  and  the  contract  shall  be  modified 
In  writing  accordingly... 


...Until  such  modification  Is  made,  the  Contractor 

shall  not  be  obligated  to  continue  performance  or 

Incur  costs  beyond  the  point  established  In  the 

clause  of  this  contract  entitled  "Limitation  of 

Cost"  or  "Limitation  of  Funds". 

• • » 

In  this  vein,  the  Board  notes  In  SteaCuzg  Pteccaion  Cotp.,  ASBCA  Ho.  4646. 
59-2  BCA  2371  that 


...The  cardinal  difference,  albeit  not  the  only  one, 
between  the  fixed  price  contract  and  the  CPFF  type 
of  contract  before  us  Is  that  under  the  latter,  the 
contractor  Is  not  obligated  to  continue  performance 
or  to  Incur  costs  Is  excess  of  the  cost  estimated 
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'^beforehand  and  stipulated  In  the  contract.  The 
' I'^effwt  of  this  provision  general  1y»  if  the  Government 
does  not  provide  additional  funds*  is  to  excuse  the 
contractor  from  failure  to  perform  under  the  CPFF 
clause  entitled  "Excusable  Delays",  ^ 

'•  . ^ ^ • j i ■ 

* . •••  ''jT 2^! 


A 

r 


yi  i { 


: 


The  changes  clause  makes  two  points:  * ' 

1«  negotiate  an  equitable  agreement  in  cost  and  fee*  and 


> I 


2.  the  contractor  Is  not  required  to  perfoni  or  Incur  costs  beyond  the 
estimated  cost  set  forth  In  the  Schedule,  which  Is  referred  to  by  the  Excusable 
Ociqys  clause.  * rv  .... 


The  clause  Itself,  and  the  Board's  decisions  lead  Into  the  other  two  clauses. 


c.  The  EXCUSABLE  DELAYS  clause,  ASPR  7-203.11,  uses  such  terarinologor  as  to 
generally. excuse  the  contractor  from  termination  for  default  because  of  failure 
to  perform  the  required  work.  '•i.?  ^ . 


L-/ 


...the  Contractor  shall  not  be  In  default  by  reason  of 
any  failure  In  performance  of  this  contract  In  accord- 
ance with  Its  terms  (Including  any  failure  by  the  Con- 
tractor to  nuke  progress  In  the  prosecution  of  the 
work  hereunder  which  endangers  such  performance)  If 
such  failure  arises  out  of  causes  beyond  the  control 
and  without  the  fault  or  negligence  of  the  Contractor. 
Such  causes  may  Include,  but  are  not  restricted  to: 
acts  of  God  or  of  the  public  eneiny;  acts  of  the  Govern- 
ment In  either  Its  sovereign  or  contractual  capacity; 
fires;  floods;  epidemics;  quarantine  restrictions; 
strikes;  freight  embargoes;  and  unusually  severe 
weather;  but  In  every  case  the  failure  to  perform 
must  be  beyond  the  control  and  without  the  fault 
or  negligence  of  the  Contractor. 


As  a result,  where  terminations  are  used  at  all,  they  are  normally  terminations 
for  convenience. 


■ d.  This  brings  us  to  the  LIHITATION  OF  COSTS  or  FUNDS  clauses,  primarily 
ASPR  7-203.3  and  402.2.  These  clauses  are  similar,  the  salient  points  of 
which  are  set  forth  In  the  following  three  quotations: 


(• 


It  Is  estimated  that  the  cost  to  the  Government  for 
the  performance  of  this  contract  will  not  exceed  the 
estimated  cost  set  forth  in  the  Schedule,  and  the  Con- 
tractor agrees  to  use  his  best  efforts  to  perform  the 
work  specified  in  the  Schedule  and  all  obligations  under 
this  contract  within  such  estimated  cost... 


i I 


i - 
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Provisions  for  the  contractor  to  notify  the  Government  concerning  projected 
overruns  are  also  made  by  the  clauses.  Terminology  differs  when  the  contract 
Is  fully  funded  or  Incrementally  funded,  but  the  following  Is  Indicative  of 
the  language:  _ 

• * ^ * V* 

• - 

...If,  at  any  time,  the  Contractor  has  reason  to 

believe  that  t<)e  costs  which  he  expects  to  Incur 

In  the  performance  of  this  contract  In  the  next  - j 

succeeding  sixty  (60)  days,  when  added  to  all 

costs  previously  Incurred,  will  exceed  seventy-  .i' 

* five  percent  (75X)  of  the  estimated  cost  then 

' set  forth  In  the  Schedule,  or  If,  at  any  time, 
the  Contractor  has  reason  to  believe  that  the  " •? 

total  cost  to  the  Government  for  the  performance  ■ ■ ' '.','•1,  . 
of  this  contract,  exclusive  of  any  fee,  will  be 
.greater  or  substantially  less  than  the  then  ^ t 

estimated  cost  hereof,  the  Contractor  shall 
>:  notify  the  Contracting  Officer  In  writing  to  . . • • ,.i 

■ t-that  effect,  giving  his  revised  estimate  of  - • ‘ ‘f; 

such  total  cost  for  the  performance  of  this  ' ’ ’ ’ 

contract.  , 

The  clause  also  Includes  a section  on  the  obligations  of  the  parties  that  Is 
similar  to  the  phraseology  used  In  the  changes  clause  discussed  above. 

...the  Government  shall  not  be  obligated  to 
, reimburse  the  Contractor  for  costs  Incurred 

In  excess  of  the  estimated  cost  set  forth 
In  the  Schedule,  and  the  Contractor  shall 
not  be  obligated  to  continue  performance 
under  the  contract  (Including  actions  under 
the  Termination  clause)  or  othen.ise  to 
Incur  costs  In  excess  of  the  estimated  cost 
set  forth  In  the  Schedule,  unless  and  until 
the  Contracting  Officer  shall  have  notified 
the  Contractor  in  writing... 

The  purpose  of  the  clause  Is  thoroughly  described  In  UtUmcheZ  EnaineefuM 
Co.,  Inc.,  ASBCA  No.  7046,  62  BCA  3348  as  follows: 

...This  clause  is  designed  to  give  the  Govern- 
ment unilaterally  an  effective  tool  to  prevent 
the  overexpenditure  of  appropriated  funds  by 
establishing  the  estimated  cost  as  the  limit 
of  the  Government's  obligation  to  make  payment, 
but  providing  a method  whereby  the  Government 
may  at  Its  election  Increase  the  estimated 
cost  and  authorize  the  contractor  to  continue 
performance.  When  the  contractor  complies 
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with  the  Limitation  of  Cost  clause,  it  protects 
the  contractor  fully  against  incurring  cost  for 
which  it  cannot  be  reimbursed,  and  at  the  same 
time  It  provides  advance  information  to  the 
Government,  so  that  additional  funds  oiay  be 
made  available  In  ample  time  to  prevent  ces- 
sation  of  work.  If  Government  officials  charged 
. with  such  responsibility  decide  that  the  allot- 
'’ment  of  additional  funds  to  the  project  Is 
warranted.  A contractor  has  no  right  to  usurp 
the  Government's  prerogative  to  control  the 
expenditure  of  public  funds  by  violating  the 
* Limitation  of  Cost  clause  ai^  continuing  *o 
Incur  costs  after  the  funds  ’'earmarked'  for  the 
project  have  been  exhausted... 


• • 'i  y-t 


. ^ .-fra  *4. • ft 
1 ^ . 

: ^ - li  , 


The  board  has  generally  held  the  Contractor  to  the  terms  of  the  notification 
requirements.  In  PRP  UzcJtnonijcA,  Jnc.,  ASBCA  No.  7713,  62  BCA  3282,  the 
/ Board  strictly  Interpreted  the  clause. 


u 

O 

u 


...He  do  not  question  appellant's  good  faith. 
, v.0n  the  other  hand  'while  the  results  may  be 
harsh  we  have  no  choice  other  than  to  enforce 
the  contract  as  written.  The  terms  of  the 
Limitation  of  Cost  clause  are  quite  plain. 
Appellant  h^d  a duty  to  notify  the  Government 
when  it  reauied  the  85*  figure  named  in  the 
contract,  and  as  a matter  of  prudent  business 
It  must  have  known  or  should  have  known  what 
Its  actual  situation  was  cost-wise  at  all 
times  during* performance. 


'b 


I I 

e.  Cost-plus -a -percentage-of  "Cost  (CPPC)  contracting  systems  are  prohibited 
under  several  statutes  that  are  not  necessary  to  discuss  here.  Generally,  the 
prohibition  applies  to  the  contract  itself,  or  a contract  provision  that  makes 
the  apparent  contract  type  a CPPC  contract  in  fact.  However,  the  Board  has  also 
held  in  AmtfUaui  Pipt  6 Stzxt  Coap.,  ASBCA  No.  4611,  60-2  BCA  2769  that  certain 
equitable  adjustment  calculations 


would  be  subject  to  the  justifiable  suspicion 
that  the  contract  had  been  converted  into  an 
illegal  cost-plus-a-percentage-of-costs  contract. 

A further  discussion  of  CPPC  in  the  area  of  concern  to  this  ..aper  is  found  in 
paragraph  4c  below. 


f.  To  summarise  the  discussion  on  background; 


JP*  Government  and  a contractor  have  entered  Into  a contractual 

the  contractor  promises  to  use  his  best  efforts  to  accomplish 
funds  allocated  by  the  amtract.  The  contract 
o?the^2**  * **  ^ Government  upon  perfoimnce  * 

. ' . * • ■ •.  > -^v,  .'.r  . 

(2)  Ho»«ver,  the  contractor  does  not  promise  to  accomplish  the"^ work 
irithin  the  fun*  provided.  Nor  does  the  Government  promise  to  provide 

ifii  S"!!LIlI!r**  ^ contractor  finds  during  performance  that  more  funds 
will  be  needed.  In  fact,  the  contractor  cannot  usually  be  held  at  fault 

provided  the  failure  to  perform  was  not  his  fault 
and  was  beyond  his  control. 

^ iV  *«*1ng  performance  of  the  contract,  the  Contracting  Officer 
tthi authority,  the  parties  must  negotiate  an 
suitable  adjustment.  This  agreement  will  Include  appropriate  Increases  or 

SSIuiSs  sfchedule,  fee,  or  other  "feEted  ronSaS 


.1 
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o, 
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. 
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vr. 
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**,  the  DECISION:  With  the  underlying  philosophy  thus  7'^'  -^. 

dlMussed^*  outside  of  normal  contract  performance  should  be  ' •*'  ^'Y . ,V 

' '■*  ''  ■ "i**.},  • *.;.T 


a.  ^ con^ctor  properly  notifies  the  Government  of  an  Impending  overrun 
In  co^>11ance  with  the  Limitation  of  Cost  or  Funds  clause  of  the. contract,  or 

a change^rder"*w'^^"^  oT«cer  unilaterally  exercises  his  authority  and  Issues 

negotiate  a change  to  the  contract  by  a supplemental 


or 


there  is  late,  defective,  or  Inappropriate  Government  furnished  property, 


chanE;  t^thE  EStJJEt?^  circumstances  leading  to  the  need  for  a • 

Decoding  upon  the  sitjatlon  that  exists,  there  are  several  avenues  or  alter- 
nates, for  resolving  the  situation: 

r®*'  “"''en^ence-  Exercising  this  alternative  could  mean 
tnat  the  Government  no  longer  has  a need  for  the  effort  being  procured; 

6.  allow  the  contractor  to  continue  to  work  and  expend  funds  up  to  the 

Schedule.  The  parties  must  then  reach  an  equitable 
agreement  as  provided  by  a paragraph  of  the  Limitotlon  of  Cost  or  Funds  clauses; 

c.  fund  the  overrun.  This  normally  Involves  a negotiation  to  establish 
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the  Mount  of  the  overrun  and  thus  detemlne  the  amount  to  be  funded.  Deciding 
to  follow  this  alternative  means  that  the  Government  still  wants  what  It 
originally  contracted  for: 


' d,  reprogrM  or  redirect  the  contract  to  reduce  the  work  and  the  estimated 
total  cost  and  fixed  fee.  This  normally  entails  revision  of  a portion  of  the 
Government  requirement*  often  a part  of  the  statement  of  work; 

I 

e.  reprogram  or  redirect  the  contract  to  Increase  the  estimated  total 
cost  and  fixed  fee.  This  usually  requires  adding  scxne  requirement  In  quantity 
of  uiilts  to  be  delivered,  tightened  delivery  schedule.  Increased  testing,  etc. 


any  combination  of  e through  e. 


.1 


The  terms  reprogram  and  redirect  are  used  Interchangeably.  For  the  sake  of 
discussion  and  complete  Illustration  of  the  method  presented  by  the  forms  to 
be  discussed  later,  proper  notification  of  an  Impending  overrun  will  be 
assumed.  Of  the  alternates  presented  above,  alternate  i Is  selected.  It 
provides  the  most  complicated  and  therefore  best  alternate  for  presentation  of 
the  method  and  an  example. 


3.  THE  SOLUTION:  The  contractor  properly  notified  the  Contracting  Officer 
that  an  overrun  of  the  total  estimated  costs  set  forth  In  the  Schedule  was 
forecast.  The  Government  knows  the  amount  because  the  contractor  Is  caused  to 
provide  this  Information  by  the  Limitation  of  Cost  or  Funds  clause.  The  act 
of  funding  does  not  take  place  at  this  point  however.  The  Contracting  Officer 
must  first  make  a finding  of  fact  as  to  the  cause  of  the  overrun.  If  he  finds 
•that  the  overrun  was  beyond  the  control  and  without  the  fault  or  negligence  of 
the  contractor,  an  analysis  of  the  reasonableness  of  the  propsed  overrun  costs 

Is  required.  The  Limitation  of  Cost  and  Funds  clauses  state: 

* • ^ ' 

If  at  any  time  the  Contractor  has  reason  to 
believe  that  the  costs  which  he  expects  to 
Incur  In  the  perfonnance  of  this  contract  In 
• • the  next  succeeding  sixty  (60)  days,  when 

added  to  all  costs  previously  Incurred,  will 
exceed  seventy-five  percent  (75t)  of  the  total 
amount  then  allotted  to  the  contract,  the 
Contractor  shall  notify  the  Contracting  Officer 
In  writing  to  that  effect. 


This  sentence,  by  stating  that  the  previously  incurred  costs  should  be  added 
to  those  the  contractor  expects  to  incur,  sets  forth  the  only  method  that 
should  be  used. 

a.  I developed  AFFTC  Form  69,  Current  Contract  Fiscal  Status  Report  to 
provide  the  data  necessary  to  evaluate  an  overrun.  The  form,  provided  as 
Exhibit  1,  Is  fully  explained  in  the  Instructions  on  the  reverse  of  the  form. 
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This  analysis  Is  to  be  based  on  the  contract  as 
currently  written  and  should  not  reflect  any 
-anticipated  changes  to  the  Covemment's  requlre- 
■ents.  Completion  of  this  fora  presents  the 
anticipated  vaiue  of  the  contract  as  written 
(estimate  at  completion)  and  Identifies  the 
extent  of  any  overrun.  Establishing  the  estimate 
at  completion  and  extent  of  the  overrun  Insures 
that  a.or  unintentional  *Cost-p1us>a-percentage- 
of-cost*  calculation  will  be  avoided  when 
changes  to  the  work  are  negotiated.  Completion 
of  this  form  allows  the  Government  to  make  the 
decision  whether  or  not.  or  at  what  level  to 
fund  the  overrun.  . ..  -r  . 
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Following  the  guidance  set  forth  by  the  Limitation  of  Cost  and  Funds  clauses  • 
discussed  above,  the  form  provides  for  six  colunns: 


A.  Cbst  Element:  The  contractor  Is  to  propose  In  DO  Form  633  type  format. 
This  provides  uniformity  and  ease  of  evaluation.  . . . > 


U 

O 


B.  Cost  Incurred  to  Date:  Thece  Incurred  costs  are  those  costs  the 
contractor  has  experded  In  performance  of  the  Government's  requirements.  Since 
the  contractor's  accounting  system  must  be  approved  by  DCAA  for  enst  reim- 
bursement contracts,  the  costs  presented  In  this  colunn  are  readi  / verifiable 
by  the  ACO  or  a member  of  his  staff.  t. 


C..  Estirate  to  Complete:  This  colicm  reflects  the  contractor's  current 
estimate  of  the  effort  necessary  to  complete  vhe  uncnopleted  portion  of  the 
contract  as  written.  The  form  Instructions  provide: 


This  column  presents  data  crucial  to  successful 
analysis  and  negotiation.  Provide  cost  or 
pricing  data  In  sufficient  detail  supporting 
the  estimate  to  show  that  current  labor,  over- 
head, material,  etc.,  rates  and  factors  have 
been  utilized,  and  that  labor  effort  proposed 
Is  adequately  doc’<nented.  Use  of  current 
rates  and  factors  In  calculation  of  this 
estimate  Insures  that  any  overrun  Is  shown 
In  current  dollars,  and  that  any  funding 
action  will  cover  the  actual  extent  of  overrun. 


I It  Is,  of  course,  essential  that  this  column  be  composed  of  the  most  current 

- estimates  available.  Using  arv  data  that  Is  less  than  current  will  result  In 

.underfunding  or  overfunding  the  overrun. 

* 0.  Estimate  at  Completion  (sometimes  also  called  the  Revised  Total  Estivated 


Cost  at  Completion):  This  column  Is  the  sunnatlon  of  costs  Incurred  to  date. 


iiiiiiliiirtifliffllilfilf 


plus  the  estimate  to  complete  the  uncompleted  portion  of  the  contract  as  currently 
written.  As  shown  In  Exhibit  1,  fee  really  only  becomes  a factor  at  this  point. 
Entering  fee  in  the  appropriate  place  In  this  column  again  puts  the  contractor 
on  notice  that  fee  Is  not  paid  on  overrun.  If  the  overrun  was  not  caused  by  the  1 
Government.  The  Board  has  held  that  where  the  Government  was  the  causative  agent, 
an  equitable  fee  adjustment  Is  due. 

E.  Current  Negotiated  Contract  Amount:  ' The  contractor  enters  the  current 
contract  cost  elements  and  fixed  fee  as  presently  negotiated. 

F,  Column  F:  This  colunrn  Is  the  difference  between 'column  E and  colimn  0 
and  provides  In  clear  detail  the  underrun  or  overrun.  The  colunn  Instructions 
require  that  the  contractor  Include 

narrative  docwentatlor  which  supports  reasons  for  the  , ' 

variance  Including  technical  differences,  material 
escalation,  rate  escalation,  etc.’  This  colunn  must 
be  supported  by  a 00  Form  633-4.  If  the  amount  of  : 

funded  overrun  exceeds  $100,000,  a certificate  will  \ . 

. be  required. 

- .® 

Note  In  the  Exhibit  that  a zero  (0)  Is  entered  for  fee  since  this  Is  the  norml 
condition  for  fee  In  this  coluom.  A 00  Form  633  series  form  and  certificate  of 
nmnt  cost  or  pricing  data  are  required  In  order  to  comply  with  ASPR  3-807.3  [ 


A Form  69  should  be  used  In  every  case  to  support  a contractor's  claim  for 
overrun.  It  Is  the  simplest,  yet  most  complete  method  for  obtaining  the  data 
necessary  to  evaluate  overruns,  or  to  ascertain  the  present  condition  of  the 
.contract.  ^ . 

' * * • — ^ u.  . ..  . . 

b.  To  continue  the  discussion  of  the  method,  alternate  { was  chosen, 
which  Is  a combination  or  occurrances:  overrun,  adding,  and  deleting  effort. 

Me  have  discussed  overruns,  recognizing  that  they  occur  and  that  the  Goverment 
may  or  may  not  fund  them.  Additions  and  deletions  of  work  required,  accelerating 
or  decelerating  schedules,  changing  specifications,  and  others  are  all  occurences 
that  happen  frequently.  The  problem  revolves  around  the  proposal  of  the  Increased 
or  decreased  costs,  and  the  resolution  of  any  fee  Increases  or  decreases  asso- 
ciated with  the  cost.  The  AFFTC  Form  70,  Contract  Change  Proposal,  was  developed 
to  fo11o*w  the  nethod  used  by  the  Board  and  to  cope  with  the  Inadequacies  of  the 
DO  Form  633-5  In  the  cost  reimbursement  area,  which  will  be  discussed  later. 

It  Is  provided  In  Exhibit  2.  The  Instructions  on  the  reverse  of  the  form 
provide  this  explw-«at1on  of  Its  purpose:  , 

Completion  of  this  form  provides  the  basis  for  successful 
negotiation  of  any  overrun  and  addition/deletion  charges 
effecting  the  contract  as  currently  written.  AFnc  Form 
69,  Current  Contract  Fiscal  Status  Report,  should  be 
completed  before  this  form  Ithe.  Foam  70}  to  present 
the  extent  of  any  overrun,  and  the  estimate  at  completion. 

Negotiation  of  these  two  Items,  especially  the  estimate 
at  completion,  allows  the  negotiations  to  continue  on 
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w aaauions/deletions  by  establishing  the  estinate 
at  completion  as  a firm  baseline. 


. I 

i). 


Tlie  coluans  are  described  below: 

Cctfict'ris'cfllSJi, 

I 

?*  dt  Cofopletlonz  The  ddte  entered  here  Is  the  saw  as  thaf 

In  }kMkXK  Food*,  Inc.,  ASBCA  No  2090.  57-1  BCA  1229.  the  Board  rl  ‘ 

♦ • r ' r* 

suitable  adjustment  is  the  difference  i 
oeween  wnat  ft  would  have  reasonably  cost  to  perform 

• Xf*  * «s  originally  required  and  what  it  reaconablv  ( 

1 cost  to  rerforw  the  work  as  changed.  “ * - I 

Therefore,  it  is  proper  that  this  colum  (B)  be  presented  first.  ! j 

4.  ♦k*  12^1*5®^  entered  in  this  column  is  the  «^ai*  as  that  shown 

S!  to  ^ Deleted'  column  of  the  00  Form  633-^  ThTiSstrUJiSST 

on  the  Fom  70  parallel  those  of  the  633-5.  but  are  more  explicit: 

Enter  in  Column  C work  that  has  been  deleted,  and  is 
not  yet  performed.  This  deleted  effort  must  be  taken 
out  at  current  estimated  hours  and  rates  to  maintain 
^tinuity.  Your  calculation  of  the  estimate  to  complete 
**  currently  written  on  the  previous  form 
Included  deleted  work  not  to  be  performed  calculated  at 
current  rates.  To  now  remove  this  work  at  current  rates 

problems.  Since  the  goal 

te^  which  negotiation  of  the  data  provided  by  you  on 
is  a plus  or  minus  funding  action 
In  today  s dollars,  a valid  estimate  using  current  data  is 
essMtiai.  AH  deleted  costs  shall  be  based  on  current 
*k  would  have  been  to  complete 

^ delet^  work  as  of  the  time  the  change  was  contemplated. 

That  is,  if  all  or  a part  of  the  work,  to  be  deleted  was 
in  an  overrun  condition,  your  estimate  must  provide  the 
overrun  costs  calculated  at  current  rates. 

Sed*  "thS  critical  that  the  most  current  data  be 

0.  Added  Cost:  The  contractor  must  enter  costs  for  added  work  which  is 
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not  presently  Included  In  the  statement  of  work.  Docunentatlon  requirements 
for  this  column  are  to  be  treated  the  same  as  it  would  be  in  a new  proposal. 
The  data  entered  In  t^is  column  must  be  the  same  as  that  shown  in  the  'Cost 
of  Work  Added'  column  of  the  633-5. 

E.  Revised  Estimate  at  Completion:  This  is  the  new  contract  amount  after 
the  required  changes,  and  includes  any  funded  overrun.  The  equitable  adjust-, 
ment  of  cost,  as  noted  in  the  UodtAn  Foods  case  above  is  column  B minus  ' 
coluan  C plus  column  0.  resulting  in  column  E.  This  approach  successfully 
avoids  any  reference  to  the  originally  negotiated  costs  as  the  Board  has 
indicated  wc  should  do  in  Jcaay  Fine  eC  at.,  ASBCA  No.  9101,  1964  BCA  4378. 

The  total  cost  approach  has  been  clearly  dis-  ' • ■ *' 

credited  as  an  acceptable  measure  of  compensa-  ' * : ' 

tion  in  as  much  as  it  assumes  the  accuracy  of  the 
.0=1  original  bid...  » , . , ^-.is 


L 


F.  Coluim  F is  self  explanatory,  and  is  the  Current  Negotiated  Contract 
Amount.  , ^ ^ 

i f ' ^ . S 

6.  Funds:  However,  we  still  need  to  determine  how  much  this  equitable 
adjustment  in  cost  adds  or  subtracts  to  the  present  contract.  To  accomplish 
this,  subtract  column  F.  the  Current  Negotiated  Contract  Amount,  from  column 
E the  new  contract  face  value,  or  Revised  Estimate  at  Completion. 

c.  This  coepletes  the  discussion  of  equitable  adjustment  with  regard  to 
cost.  The  method  presents  a quick,  simple,  accurate  and  complete  format  and 
procedure  for  fully  discovering  overruns,  providing  for  added  and  deleted  effort, 
and  insuring  that  only  the  funds  needed  are  provided  or  reduced  as  appropriate. 

d.  Resolution  of  Fee:  The  fixed  fee  may  be  increased  or  decreased  depending 
on  work  accomplished.  ASPR  3-405.6.  previously  referenced,  provides  for  this. 

The  Changes  clauses  provide  for  an  equitable  adjustment  In  the  amount  of  fee 

to  be  paid.  The  ASBCA  addresses  this  In  numerous  cases.  In  H.K.  Fvtguson  Co., 
ASBCA  No.  2826.  57-1  BCA  1293.  the  Board  said  about  additive  changes: 

The  fixed  fee  is  the  contractor's  compensation 
for  doing  the  work  prescribed  by  the  contract, 
that  Is.  the  work  contemplated  by  the  parties 
when  the  contract  was  negotiated  and  on  which 
the  fee  was  based.  The  amount  of  work  cannot 
be  increased  materially  without  an  adjustment 
of  the  fee... 


I'ji  •- 


Concerning  deductive  changes.  It  said  in  EmeJuon  EUctfUc.  Co.,  ASBCA  No.  15591. 
72-1  BCA  9440  that 


He  are  inclined  to  the  view  that  Item  III 
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represented  j significant  part  of  the  total  / k 

perfomance  called  for  by  the  contract,  with  . V , ^ 

the  result  that  the  el Ininatlon  of  such  wort 
fron  the  contract... should  result  In  a sub- 
stantial reduction  In  the  fixed  fee. 

In  general  the  Board  has  held  this  to  be  a consistent  approach  to  changes. 

It  suted  this  In  Roacoe  Eng-uteea^  CoJtp.,  ASBCA  No.  5370,  61-2  BCA  3148: 

a... profit  factor  exist  generally  be  added  to 
the  contractor's  costs  whether  the  change  In  . • 

work  Is  additive  or  deductive.  This  Is  ‘ ‘ 

consistent  with  the  Board's  ruling  In  other 
Instances...  . •!  ' •,  . 

* m \ 

Relative  to  the  base  for  fee  calculation  the  Board  In  HcAbeJU,  Inc.,  ASBCA  Ko. 

3283,  57-1  BCA  1319  said  that  . . w . 

• * » ' 

Whatever  way  have  been  appellant's  hopes  and 
fears  concerning  profit  and  loss,  the  fact 
’ remains  that  In  computing  profit  on  the  • • 

basis  of  cost  the  contracting  officer  was 
applying  an  established  principle  of  accounting 
and  we  can  find  no  Justification  from  the 
available  evidence  In  the  record,  as  dis- 
tinguished from  mere  allegation,  for  refusing 
to  adopt  It.  . . . • 

'It* should  be  noted  that.  In  the  Hvtbtfit  case  above,  the  Board  used  the  same 
profit  percentage  as  was  originally  negotiated  In  calculating  the  reduction. 

There  Is,  however,  no  requirement  to  do  so,  as  follows.  In  KtqtAnon  BlcuA,  Inc., 

ASBCA  No.  10738  , 65-2  BC/i  5127,  the  Board  reiterated  Its  general  philosophy 
and  added  a proviso: 

The  ^ard  hereby  reaffirms  these  decisions 
holding  that  the  same  principles  used  in 
pricing  additive  changes  apply  generally 
to  deductive  changes,  giving  appropriate 
consideration  to  special  circunstances 
which  would  reduce  the  credit  otherwise 
due  the  Government. 

Specific  examples  are  taken  from  H.K.  Feaguaon  Co.,  cited  above: 

Having  arrived  at  the  amount  of  Increased  cost 
resulting  from  changes  Increasing  the  amount 
and  character  of  the  work,  we  now  have  the 
problem  of  determining  the  a-vkun^  of  adjustment 
In  the  fixed  fee.  The  fee  is  'isually  computed 
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as  a percentage  of  the  estirated  cost,  because 
It  has  been  found  fn»  experience  that  the 
estlaated  cost  provides  a practical  and  satis* 
factory  basis  for  arriving  at  a reasonable 
fee.  although  the  percentage  figure  used 
depends  on  several  factors  such  as  the  type 
of  work  and  services,  the  dollar  amount,  etc. 


and  from  Keho  InduUnlu,  Inc..  AS8CA  5762  . 60-2  BCA  2738: 


As  a 


■otwithstanding  appellant's  assertion  to  the 
contrary,  difficulty  attending  performance  of 
a contractual  requirement  Is  a major  consideration 
In  the  determination  of  the  amount  of  profit 
to  be  allowed  a co’'tractor...  . r i.:  - ; 

;■  r-ic 

result  of  the  cited  cases.  •!...<  n.j» 


> 1.  cost  Is  the  acceptable  fee  base  for  both  additions  and  deletions; 

2.  The  negotiated  fee  percentage  Is  acceptable  as  the  baseline  from 
idilch  to  negotiate  a higher  or  lower  fee  amount  due  to  factors  such  as 
risk,  difficulty,  etc. 

• The  forms  provide  for  these  conclusions.  However.  I do  not  consider  that  ttie 
. . total  estimated  cost  amount  presented  by  column  C of  Exhibit  2 Is  the  appropriate 

' fee  base  In  overrun  situations  where  the  overrun  is  not  being  funded.  This 
colirnm.  Deleted  Cost.  Is  calculated  based  upon  today's  rates,  factors,  and 
hours,  and  Includes  whatever  part  of  the  overrun  Included  In  coltam  B Is  not 
( I . funded.  To  calculate  a fee  reduction  on  this  cost  will  result  in  too 

much  fee  being  deleted.  It  results  In  fee  deletion  based  on  overrun  work  that 
Is  not  to  be  funded.  It  penalizes  a contractor  tdiose  contract  only  requires 
his  'best  efforts*.  He  has  not  promised  successful  performance,  does  the 

Government  then  penalize  him  for  non-successful  performance?  I submit  that 

we  should  not.  (Please  bear  in  mind  that  1 am  discussing  Increasing  or 
decreasing  the  total  face  value  amunt  of  fee.  not  the  fee  that  may  or  may  not 
be  paid  depending  upon  success.  I recognize  that  fee  payment  Is  based  generally 
on  percent  of  completion).  What  then  should  the  fee  reduction  base  be?  It 
should  be  the  amount  that  Is  Included  In  the  present  negotiated  face  value 
for  the  deleted  effort.  If  It  Is  Inequitable  to  take  fee  back  on  overrun  dollars 
that  will  not  be  provided  and  were  never  negotiated,  then  It  should  be  equitable 
to  take  It  back  on  the  dollars  that  were  negotiated.  In  G.M.  Co.  Uanuiactufuna 
Inc.,  AS6CA  No.  2883,  57-2  BCA  1505  the  significance  of  cost  as  a base  is 
emphasized: 


There  are  many  factors  to  be  considered  In  determing  an 
equitable  adjustment  In  price  for  a change  order,  but 
by  far  the  most  Important  factor  Is  cost.  When  a 
change  order  Involves  performance  of  additional  work. 


I 
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ordinarily  the  price  adjustiKnt  Is  based  on  the  cost 
of  the  additional  Materials  and  labor  to  which  Is  added 
an  allowance  for  overhead  and  profit.,, The  sane  principles 
used  In  pricing  change  orders  Involving  Increased  costs 
apply  generally  to  the  pricing  of  change  orders  which 


reduce  the  cost  of  perforaance 


The  original  value  of  this  deleted  work,  negotiated  at  am  length.  Is  the  only  . " 
reasonable  Measure  of  Its  value  and  Is  the  only  reasonable  basis  for  fee  reduction. ' 
See  the  H.K,  FetgusoK  Co.  case,  quoted  above.  The  Instructions  on  the  for«  require 


this  approach 


To  avoid  a penalty  effect  of  rcHOvIng  fee  at  a higher 
level  than  that  negotiated,  you  May  enter  the  fee  to 
be  reonved  In  this  column  based  upon  rates  and  factors 
originally  negotiated.  However,  on  a separate  sheet, 
you  Must  show  how  the  costs  were  derived  that  the  fee 
reduction  Is  based  upon. 


4.  THE  EXAMPLE:  Exhibits  4 and  5 provide  a recent  case  which  was  successfully 
negotiated.  An  explanation  of  the  data  entered  In  the  colunns  would  be  a near 
reiteration  of  the  previous  discussion.  The  mainder  of  this  paragraph  Is  a 
discussion  of  the  00  Form  633>5,  and  the  problem  of  fee  reduction  where  there 
Is  an  overrun,  and  uncompleted  ^fort  Is  to  be  deleted. 


a.  The  first  Item  that  will  be  addressed  Is  the  00  Form  633-5,  the  primary  ( 
change  administration  vehicle  provided  to  administrators.  I have  found  that 

•this  form  Is  unacceptable  for  all  but  the  most  simple  change  problems.  It 
Is  misleading  at  best,  and  dangerous  at  worst  In  complex  overrun/redirection 
situations.  The  form  should  only  be  used  as  the  primary  vehicle  In  FFP  con-  ( 

tract  modifications.  Why?  One  significant  reason  Is  that  the  form  presents  a 
column  entitled  'Net  Cost  of  Change'.  If  this  column  Is  what  Is  claims  to  be, 
then  any  negotiator  or  buyer  should  be  expected  to  add  this  coluirm  to  the  present 
face  value  of  the  contract.  This  might  be  acceptable  for  a FFP  contract.  But, 

In  CR  or  CPFF  procurements  this  technique  Is  asking  for  trouble.  Exhibit  3, 
for  example  Is  an  extract  of  a 633-5  completed  for  a recent  situation.  On  the 
face  of  It,  It  appears  to  be  acceptable.  In  fact  It  was  acceptable,  for  the  ^ 

Government  team  evaluated  the  add/delete  proposal,  and  this  Is  the  negotiation 
result.  Exhibit  4 Is  the  completed  Current  Contract  Fiscal  Status  Report,  showing 
a $234,638  overrun.  Exhibit  5 Is  the  completed  Contract  Change  Proposal  showing 
a net  decrease  In  funding  of  $799,253,  (Column  6).  You  will  notice  that  net 
decrease  In  funding  of  $799,253  Is  not  quite  the  same  as  the  $1,047,786  Net 
Cost  of  Change  amount  shown  on  the  633-5.  One  main  reason  for  the  difference 
Is  the  overrun  shown  In  Exhibit  4,  colum  F.  None  of  It  Is  going  to  be  funded. 
Therefore,  all  of  It  Is  Included  In  the  Deleted  Cost  column  of  Exhibit  5. 

b.  The  other  reason  for  the  difference  Is  associated  with  the  fee  deletions. 
Normally,  fee  addition  Is  not  a problem,  since  It  Is  treated  as  a new  procurement 
and  negotiated  as  such.  The  fee  deletion  problem  Is  always  associated  with 
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j.  CONTRACT  PRICING  PROPOSAL  (CHANGE  ORDERS) 

I * 


Cost  Elements 


I 


Net  Cost 
To  Be 
Deleted 


Subcon.  Items 

i , V ; r.  /;} 

Material  0/H  r . 

I I 

Interdivisional 
I Jransfers 

' I 

Engr.  Labor 

. I 

Engr.  0/H 
' J • 

Mfg.  Labor 


^ 

Xr  :ir 

•*) 


Hfg.  0/H 
Other  Costs 
Subtotals  “ . 

G&A  Expenses 
ToUl  Est.  Cost 
Fee 

Total  Price 

DO  Form  633- S Extract 


Cost  Of 

Mork  Added 


r ^ . i 

Net  Cost 
Of  Change 


$ 686,371 
112,159 

4,528 
K,245_ 
4,677 
46,850 
105,521 

$■  972,351 
208,370 
$1,180,721 
69,924 
$1,250,645 


$ 30,290 
8^601 

mm" 

21,633 


U 

I'.J  4 

’ i 


8,262 
' 18,845 
43,418 
12,674 
$143,723 
45,865 
$189,588 
13,271 
$202,859 


$ • (656,081) 
(103,558) 

(4.528) 

}; 

9,388 
r ! 3,585 

^ I (28,035) 

: i 

(62,103) 

) 

i i 12,674 
$ (828,628) 
' (162,505) 
$ (991,133) 
(56,653) 
$(1,047,786) 


Exhibit  3 
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finding  an  accepUble  fee  reduction  base.  The  Instructions  of  the  633-5  require 


...*est1uted  cost  of  all  deleted  uort”  Includes 


estioates  of  what  the  cost  would  have  been 
of  the  tine  the  change  was  Issued)  to  complete 
deleted  wrk  not  yet  perforeed.  and  (11)  the  cost  of 
deleted  work  already  perfomed. 


In  ot.er  words,  the  current  estimated  cost  of  the  work  that  remains  Is  required. 
It  Is  this  present  cost  which.  If  It  Includes  an  overrun,  makes  it  unacceptable 
as  a fee  reduction  base.  The  Net  Cost  to  be  Deleted  column  shown  In  Exhibit  3 
Includes  this  $234,638  overrun.  The  633-5  Instructions  provide  no  guidance  on 
fee  reduction.  If  a contractor  follows  normal  proposal  practice,  he  will  cal- 
culate his  fee  reduction  using  the  negotiated  percentage  and  the  total  estiuted 
cost  to  be  deleted.  This  method  results  In  $69,924  fee  reduction  for  deleted 
wrk.  When  added  to  the  fee  negotiated  for  added  work,  the  result  Is  a $56,653 
fee  reduction.  This  $56,653  Is  too  much.  It  amounts  to  a reverse  CPPC  condition 


in  that  the  higher  the  contractor  calculates  the  costs  to  be  deleted,  the  greater 
his  fee  reduction  will  be.  It  motivates  contractors  to  underestimate  In  two 
areas: 


• 1.  The  estimate  to  complete.  The  greater  this  estimate  Is  beyond  the  funds  v 
remaining,  the  greater  the  overrun.  And  the  greater  the  overrun,  the  less  likely 
the  Government  Is  to  fund  It.  And  the  less  likely  the  Government  is  to  fund  the 
. overrun,  the  more  likely  It  Is  to  negotiate  a reduction  In  the  work  to  be  accomplished 
and  therefore  the  fee  to  be  paid.  And  the  greater  the  overrun  amount  Included 
In  the  deleted  cost  base,  the  more  fee  will  be  lost; 

2.  Cost  of  deleted  work  not  yet  performed.  This  is  an  estimate.  The  lower 
this  value,  the  lower  the  fee  reduction. 

Of  course  the  obvious  result  of  this  underestimating  will  be  an  underfunding 
of  the  work  remaining.  But  the  633-5,  by  not  providing  a rethod  for  an  equitable 
fee  reduction,  motivates  contractor  to  underestimate  cost  to  maximize 
the  retention  of  the  remaining  fee. 

t 

c.  Exhibit  5 shows  a fee  reduction  based  on  the  total  estimated  cost  less 
the  overrun  amount.  This  $48,275  fee  reduction  allows  $21,649  In  fee  to  remain 
on  the  contract.  This  $21,649  would  have  been  unjustifiably  removed  If  the 
overrun  had  been  Included  in  the  fee  reduction  base.  The  contractor  would  r 
have  been  penalized  that  much  for  administrative  or  technical  problems  when  this 
1$  a 'best  efforts'  procurement. 

But  would  the  contractor  actually  have  proposed  a reduction  of  $69,924?  Probably 
not.  The  contractor,  as  this  one  did,  would  not  propose  any  fee  reduction.  Or, 

If  he  did,  it  would  be  an  amount  much  scalier  than  the  $69,924  I show  in  Exhibit  3. 

Then  what  would  the  proposed  fee  reduction  be  based  upon?  Nobody  really  knows, 
except  that  It  is  the  amount  the  contractor  is  willing  to  have  deducted.  And 
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*n  "equitable"  reduction  would  have  been  negotiated.  Based  on  what?  Nobody 
knows  that  eithePt  but  Is  probably  would  have  used  the  contractor's  proposed 
reduction  as  a starting  point.  . . , , 
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Herein  lies  the  CPPC  danger.  No  one.  not  even  the  contractor,  really  knows 
what  the  fee  reduction  associated  with  the  deleted  work  should  be  without 
using  the  proposed  method.  Any  contractor  which  Is  serious  about  negotiating 
will  attempt  to  cloud  this  aspect  of  the  fee  reduction  problem.  For  even  If 
fee  Is  not  the  primary  motivator,  no  contractor  Is  willing  to  reduce  fee 
unnecessarily.  And  the  Govemcent  negotiator,  under  pressure  to  resolve  the 
problems  created  by  the  change  order  or  overrun  as  they  often  are,  wjQl_  nego- 
tiate an  equitable  adjustment.  But  sy  question  remains.  Based  on  what?  The 
Board  cases  cited  above,  especially  H.  iL  Fea^aaon,  and  Heabeat,  fully  support 
cost  as  the  reduction  base.  These  and  other  cases  Indicate  that  the  exact 
percentage  may  be  the  same  as  originally  negotiated,  or  more  or  less  depending 
upon  complexity,  risk,  and  other  factors.  It  Is  ogr  firm  belief  that  the  proposed 
percentage  must  be  the  percentage  originally  negotiated.  For  the  procurement 
as  a whole,  this  percentage  rate  was  the  rate  found  to  be  fair  and  reasonable  to 
both  parties  during  arms  length  negotiation.  It  must  be  the  starting  point. 

It  will  often  be  the  ending  point  as  veil.  But  negotiations  can  then  be  held 
to  Increase  or  decrease  the  rate  from  that  point  depending  on  the  facts  of  the 
situation.  If  necessary.  ^ ^ . ....  .. 

In  our  example  It  Is  simple  to  create  a CPPC  condition.  The  contractor  merely 
has  to  propose,  and  the  parties  negotiate  an  amount  less  than  the  $48,275  In 
fee  without  explaning  the  circumstances  making  It  different  than  the  baseline 
fee  percentage.  CPPC  enters  Into  the  picture  at  this  point:  When  not  enough 
fee  Is  returned.  The  extra  remaining  fee  may  be  considered  to  increase  the 
fee  rate  on  the  work  remaining  over  the  rate  originally  negotiated,  which  I 
believe  to  be  a potential  CPPC  system.  However,  there  Is  nothing  erroneous 
or  wrong  in  negotiating  a fee  less  than  the  baseline  percentage,  provided  that 
the  parties  agree  that  the  work  is  less  complicated  than  that  proposed,  the 
risk  on  the  contractors  part  is  less  than  on  the  renainder,  or  some  other  reason 
that  justifies  a lower  fee  reduction  and  docunents  the  basis  for  the  equitable 
adjustment.  It  is  not  enough  to  say  that  any  agreement  is  equitable  in  this 
situation.  The  facts  must  support  the  lesser  fee  reduction,  or  CPPC  results. 

The  only  subjective  aspect  of  this  equitable  fee  deletion  is  the  amount,  not 
that  one  is  necessary. 

With  regard  to  fee  adjustments,  many  cost  reimbursement  contracts  provide 
for  payment  of  fee  based  upon  percent  of  completion.  It  should  be  obvious 
that  payment  of  fee  on  this  basis,  and  removal  of  fee  based  upon  the  originally 
negotiated  costs  are  not  related.  He  are  not  dealing  with  terminations  or 
with  payment  of  fee.  The  fact  that  a contract  clause  may  require  fee  to  be 
paid  on  the  percent  of  completion  basis,  and  that  the  ACO  may  have  overpaid 
fee  normally  has  no  bearing  on  deleting  fee  for  work  which  is  not  going  to 
remain  in  the  contract.  After  deleting  fee  by  the  method  presented  by  the 
forms,  the  Government  may  find  that  a fee  overpayment  has  occurred,  and  It 
may  be  resolved  at  that  time. 


( 


150 


owrrun  exists,  ttts  approach  penalizes  a contractor  unjustly  Further  the*** 
aTthJ^Sr  *!f‘^  n«90tiator  must  use  the  originally  regotiated  percentage  rate 

proposed  fee  reduction.  ^If  other^factors^are  ^ 
an^iitt»hii*.^*^Ll*?  percentage  may  be  Increased  or  decreased  to  result  In 

SpcL2rfJIJ„  ? '■*''•  “*  Sovemwit  treadi  the  thin  1m  S 

U It  |."»i  M “t“J2t!:‘;nJ'M'?J"t^rjM^  TheSl^'LVUit 

::r  Snm's's.”"’'  »* 

4 » 

proposal,  analysis,  and  negotiation  of  CR  and  CPFF  flsrai 
problems  In  000  Is  relatively  unsupported  by  formal  guidance  Contractors  and 
Government  negotiators  use  several  different  methods  In  ^Shlng^SSSw 
that  appear  reasonable.  Oue  to  the  00  Form  SSsSTcontrntSrs  JrTSftS  Wed 

■t  all.  Therefore  actual  fee  reductions  may  be  docmented  bv  catch  words  and 
J**®  ^»tr  and  reasonable  fee  rate.  If  it  Is  used  at  all 

work^  ^ * MSt  base  that  may  not  represent  the  value  of  the  deleted 

Govcrrjient.  • Since  the  baseline  cost  and  the  fee  rate  are  ciwief'ifnAc 

IdiulSnr^***  **'“  a>PC  conditions  may  Ixlst  and  the^itable 

any  werJlS!**^^  establishing  the  current  sUtus  of  the  contract  and  Identifying 

Identifies  ^ additions  and  deletions  of  cost  against  uhat  the 
contract  would  cost  to  complete  without  changes;  samsi  mac  tne 

c.  providing  for  fee  reductions  based  upon  a non  '““naltv  orlMtod  ro«f 
base  and  the  original  fee  rate  as  equitably  Sjusted  d 

as  changS?**^^^  Provides  for  the  actual  funds  necessary  to  complete  the  contract 

exception  that  use  of  the  method  presented  bv 
clarifies  and  simplifies  the  administrator's  job.  Complex  contractual 

resolved.  Uniformity  of  approach  and  meth^ 
thoroughly  familiar  with  the  procedure.  Kego- 
iiSnahL^?  concluded  more  quickly  since  coninon  ground  for  discussion  if 
M Contractors  can  be  assured  of  a consistent  approach 

^ ?r  Government  administrators,  offices,  divisions,  or 

InadvIrLnt  j^so^“tion  of  similar  problems.  Management  can  be  assured  that 
Inadvertent  and  therefore  untraceable  mistakes  are  not  made  in  critical  cooolex 
contract  changes.  And,  It  will  make  equitable  adjustments  trCel/eqSlJibir 
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J FOR  DETCRMimWG  RELATIONSHIPS  BETWEEN  iUTBmi. 

PRICES  MD  BLS  INDEXES 


... 


Alvin  W.  Platt 

; 'Itesearch  Associate.  Graduate  School  of  Business 
V^'UalversIty  of  Sart*  Clara 


“‘.<1 


r-jr-H-.- 
i::  ' 


( 


'wilw^f  we";2‘;f'?S^s2bjS.*””*^’^  «><*  gives 

, * j 

techIiqi«T^J!}i2f^r:5d^^  Sii^te  tie  USh  :;4m  ^raElJl"® 

/ analysis  of  the  relationship  between  prices  paid  by  an  Industrial  con- 

^-^Indwes/**'^  naterlals  and  published  Bureau  of  Labor  Standards  (BLS)  type  pr1« 

'^«hr^SLl?5  ‘^f****:  of  price  levels  for  the  US  econr^  has  reached 

* *^*on?ially  higher  levels  when  compared  to  rates  experienced  after  UWIi  IhHI 

procurewnt  of  defense  and  aeresprce  Itens.  the  Federal 
”^Tk  contractual  arrangements  extending  over  a period  of  * 

utlllMH^^^nv  n ^ in»>enent  prublem  In  this  indexing  technique  Is  the 

-«h1?  p^iinr.  ?:u^'  r 

- «»»<otPpy  ».d  for  sodi  c<..™soni  " £!pl"flI3  Ihe  rej!?;.  Jf  ™. 

p ng  ac  ual  prices  and  index  data  using  the  nethodology  are  then  presented. 

on  Sl::t)le  linear  regression  techniques.  Linear 
rail  Hafyc^*k“"  ^ conducted  on  the  raw  cost  and  Index  data,  or  a^ifled  * .-n 

vaiy^lth  th?Jodified*rf»fl^f"^'  1°^  oent  change,  etc.  Regression  results 

J«<IIn  *^*5*  format  e^rloyed.  Additionally,  the  special  "no-intercept" 

wrslon  of  linear  regression  can  be  eaployed  to  eliminate  the  Intercept  coefficient  fmm 

2MI®r”  "solts  <0  the  y l ITVoZ^.  CrtteS  SvIlS- 

..  to  evaluate  the  degree  of  relationship  between  prices  and  .eyexes  are  then  explained^ 
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Results  fro*  analysis  of  actual  raw  material  prices  and  BLS  indenes  using  the 
described  methodology  are  recapped.  Analytical  results  are  then  evaluated  using  ‘ 
the  developed  criteria. ‘ Finally,  suitability  of  this  methodology  and  criteria  for 
various  conditions  and  circumstances  is  discussed.  i . , .1^1 
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ESTIMATING.  PMC1II6.  AW  KEfiOnATlNG  - A Igy  FMWTIEK 


Jaoes  R.  Brennon 

Chief.  Industrial  Management  Division 
US  Amgr  Aviation  Systems  Coamand 


r?  Sa.it  n-sz  * 


It  is  expected  the  new  frontier  In  pricing  will  offer  s 
little  staetblng  for  everyone.  For  those  sctively  Involved  in 
■sjor  weapon  system  estimating  with  specialized  eHils  in 
operation  research.  Industrial  engineering,  accounting,  price/ 
cost  analysis,  flnsmclal  management,  auditing,  production 
ccntrol,  and  contract  management,  the  real  doer  Involvei^nt 
will  be  addressed  la  chls  presentation.  For  those  with  a more 
passing  Interest,  the  detailed  discussion  will  be  rendered  into 
discrete  Issues  related  to  psst.  present,  and  future  major 
weapon  system  cost  estimating.  Lastly,  top  management  should 
find  a fresh  opportunity  to  continue  to  "do  something"  about 
the  two  specific  factors  Identified  and  associated  with  the 
accuracy  of  major  weapon  systen.  cost. 

To  those  familiar  with  MIL  Standard  881  Work  Breakdown 
Structure  requirements  and  DODI  7000.2  with  Its  Cost/Schedule 
Control  Systen  Criteria  (C/SCSC) . this  presentation  wlU  review 
familiar  ground.  This  system  will  be  proposed,  in  part,  for 
opplicstion.  In  a similar  fashion.  n»g  contractor 
and  Government  coot  proposal  managers  will  find  that  treapon 
systen  and  contract  pricing  ^Ich  are  presently  done  In  a 
swmarlzed  fashion  will  be  proposed  to  be  accomplished  on  a 
massive  detail  scalA.  The  computer  specialists  will  be 
Identified  with  this  area.  Computer  hardware  and  applications 
have  advanced,  but  their  use  in  estimating,  evaluating  and 
negotiating  has  been  somewhat  Ignored  In  procurement  research, 
and  largely  unapplied  In  our  business  activities.  For  management 
PJ-osentation  presents  Improvement  1**  two  major  areas  s 


Accuracy  of  cost  for  program  decision  purposes  where 
Increased  accuracy  of  all  portions  of  Life  Cycle  Cost  Is 
necessary  because  of  the  evolution  of  cost  as  an  equal  partner 
to  other  considerations;  and 


Accuracy  of  the  cost  base  line  for  program  cost  control 
during  a program. 


I 


/M  these  Inproveaents  uay  require  e DOD-wlde 

(including  Mntractor)  ezenlnetlon  es  to  bow  ue  will  perfon 
our  pr^sel.  eveluatlon,  end  negotleclon  efforts  for  the 

Important  Is  *,  to 

now  we  will  allocate  our  aanpower  resources  between  the  nelor 
weapon  systen  Issues  of  perfotaance.  schedule,  end  cost. 


i ^ ) 


THIEE  FACTOtS 


I 

The  technique  of  conputerlxed  estlnates  for  ewalnatlns 
proposals  has  been  developed  at  the  OS  Amy 
Aviation  SystM  Consnd  (DSAAVSCOH)  In  St.  Louis  and^llxed 
In  nuy  Should  Cost  appUcatlons.  This  presentation  draws 


h^lly  on  AVSC«'s  espenence.  Oth^;; 


•PPUcatlons.  In  fact, 

ttls  was  the  subject  of  a paper  presented  at  the  second  DOD 
rrocurcaent  Kesearch  Sy^wslua  held  la  1973.1 


The  lapleaentatlott  of  MU  Standard  881  and  Ita  systcaatlzed 
Work  Breakdown  Structure  (UBS)  established  a franeiS^ 
^tractor  acceptance  of  a cost  collection  systea  by  which  lower 
^el  estates  can  be  roUed  up  Into  the  cost  of  Mjor  end 
Itea  ^dware.  This  led  to  the  contractor's  acceptance  of  a 
detailed  lower  level  UBS  estlnatlng  systea.  However, 
systm  Is  used  after  contract  award  for  prograa  cost  and 
schedule  control. 


The  final  factor  diacoaaed  and  integrated  in  thin  paper  la 
the  capturing  of  coat  data  for  evaluation  and  negotiation  at 
the  lowest  level  of  the  UBS.  It  has  been  a unique  effort 
associated  with  Amy  coapetltlve  developnent  and  deslgn-to-cost. 


THE  PAST 


Coaputer  prograns  are  not  new  to  the  contract  pricing  area. 
There  have  been  evolving  and  laproved  coaputer  prograns  for 
the  najor  defense  contractors  or  for  specialized  uses  In  the 
early  years  of  this  decade.  Although  prograns  have  been 


^:aptaln  Grady  L.  Jacobs,  OSAF;  Coaputer  Cost  Models  In 

Pricing;  Second  DOD  Procurement  Research  Synposlun; 
Maval  Post  Graduate  School;  Honterey»  Callfomlaf  P.  180  - 208. 


155 


' f • ! 


u 

(j 


O 


o ^ 

o 

*'■) 


I 


( ) 


I 


# 


41 


1 


I 


i 


\J 

o 

u 

o 

O 


dcreloped  aad  iaproved,  ve  have  In  no  way  reached  the  place 
la  Cbelr  developaent  where  significant  Inprovenenta  are  no 
longer  poealble.  The  Initial  progrsM  nerely  conpared  the* 
contractor's  position  in  the  elenents  of  coat,  such  as  labor, 
■aterlal,  and  owerbsad,  with  the  CovezaBenc's  position.  Later 
nndels  have  the  capability  of  "**^'*"g  conplcx 
which  pemits  the  negotiation  of  tine  related  considerations 
la  the  aaterlala  and  the  labor  cost  proposed  by  the  contractor. 

The  two  tables  belov  depict  typical  ontputs  as  this  developMnt 
.took  place:  . . . . -j.  J 


1 


TABLE  1 

BABLT  mOELS 
(TALDES  Di  MILLiaKS) 


V : 


. . 

COST  ELEHENT 

PROPOSAL 

OBJECT  IVB 

■BCOndgi 

. 

Materials 

3L 

28 

24 

< 

Matl  Borden 

4 

3 

2 

u : 

TOTAL  MATL 

35 

31 

26 

Eng  Hre 

(.5) 

(.3) 

(.3) 

f 

Eng  Dollars 

3 

2 

2 

Eng  Overhead 

4 

3 

. . 3 

Mfg  Hrs 

(1.8) 

(1) 

(1.4) 

‘ Mfg  Dollars 

9 

5 

7 

1 

Mfg  Overhead 

18 

10 

14 

Other  Cost 

15 

14 

13 

- 

TOTAL  COST 

84 

65 

65 

t 

G&A 

13 

10 

U 

TOTAL 

97 

75 

76 

Profit 

10 

7 

7 

\ 

TOTAL  PRICE 

107 

82 

83 

1 

/ 
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TABLE  2 

* ^ I-.  •. » ‘ »“•-  r 

MORE  ADVANCED  MODELS 
(VALDES  IN  MnJLIOHS) 


!W‘  ^ \ r*.L  .‘•..•tf'C  yr^iy 

•/‘.jviV  I i*wr':i  Tt3ci:5  Ai 

• ' * - r,  k^JHI 


^ ^ 

> 

COST 

1975 

1976 

1977 

‘ TOTAL  . 

ELEMEVr  • . OMTR 

GOVT 

ccnmt 

COVT 

CONTR 

GOVT 

CONTR 

QDVT 

.e-  ^ 

Materials 

20 

20 

10 

8 

1- 

- 

.31 

.i!  28; 

Matl  Burden 

3 

2 

1 

1 

4 

3 

TOTAL  MATL 
• 

23 

22 

u 

9 

“T 

35 

31 

Eng  Hrs 

(.3) 

(.2) 

(.2) 

15  r**  ■ T 

(.1) 

(.5) 

(.3) 

Eng  DoUers 

2 

2 

1 

> 

• 

• 

3 

2 

Eng  Orerbcsd 

3 

3 

' 1 

f ^ 

4 

3 

•*^5  ars_ 

• . (.6) 

..(.3) 

(1.2) 

(.7) 

(1.8) 

(1) 

Mfg  Dollars 

- 

■ — 

3 

2 

6 

3 

9 

5. 

Mfg  Overhead 

6 

4 

...  12 

6 

18 

. 10 

} 

14 

65 

Other  Cost 
TOTAL  COST 

“28 

“27 

5 

27 

5 

20 

10 

29 

9 

18  . 

15 

84 

G&A 

TOTAL 

4 

32 

4 

31 

4 

31 

3 

23 

5 

34 

3 

21 

13 

97 

10 

75 

Profit 

TOTAL  PRICE 

3 

35 

3 

34 

3 

34 

2 

25 

4 

38 

2 

23 

10 

107 

7 

82 

PRESENT 
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CoKputers  are  generally  recognized  as  providing  three  special- 
ized advantages  which  are  speed,  accuracy,  and  the  related  value 
of  being  able  to  handle  great  volunes  of  material  (capacity). 

It  Is  this  latter  value  of  capacity,  which  Is  presently  under- 
utilized, that  permits  the  development  of  computer  programs  for 
the  specialized  use  In  evaluation  and  negotiation  associated 
with  major  weapon  system  source  selections  and  contract  award. 

It  Is  largely  through  the  USAAVSGOM  Procurement  and  Production 
Directorate's  effort  In  Should  Cost  that  computer  technology 
has  been  put  to  work  In  the  arena  where  the  final  act  Is  a 
signed  contract.  This  procurement  research  was  a systematic 
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' evolution,  with  no  reel  "Task  Force"  to  accelerate  developnent 
and  test.  Finally,  coaputer  prograns  have  been  developed 
which  will -faithfully  produce  the  botton  line  price  with  all 
the  accoepanylng  lower  coq>oaent  level  valnee  for  categories 
coat,  such  ae  naterlal,  nanufacturlng  labor,  wg rt 
labor,  nanufacturlng  overhead,  other  direct  chargee,  G&A; 
and  profit.  Advances  have  been  nade  to  where  the  coaputerlzed 
prograns  handle  large  anounta  of  detalled/tlna  phased  data 
for  both  Che  contractor's  position  and  the  Covemnent's 
position.  This  pemlcted  Isolation  of  the  najor  areas  of 
difference  so  that  the  parties.  In  exanlnlng  Che  conputer 
prtotouts,  could  nore  effectively  r nwiiili  si  t and  concentrate 
their  efforts  on  these  areas. 


> i rtsll  a.  rt* 


a 


FDIXT  UTILIZE  TECBEOLOGT 


i . ..  . 


The  current  USAAFSCQM  nodela.  Innovated  as  they  are  today, 

( I are  expected  to  be  outdated  as  procurcnent  estlnatlng  research 
continues  and  a new  generation  of  prograns  la  developed.  u 
These  new  prograns  will  pemlt  real  tine  negotiation  through 
f the  use  of  coaputer  science.  Whereas  Che  current  systen 

' — ''  requires  batch  processing  with  a certain  tine  lag  for  key 

punching  and  processing  of  Che  changed  Input  data,  the  progm 

U planned  for  developnent  will  pemlt  entry  via  a key- 

• board  and  negocladoos  utilizing  a display  located  with  the 
contractor  and  Govemnent  negotiators  at  the  negotiating  table. 
Such  an  Inproved  conputer  capability  provides  Che  real  time 
^ ^ capability  and  approaches  the  type  of  efficiency  ntlllzadoo 

— of  contractor  and  CovemnenC  personnel  which  la  expected  to 
sake  this  type  of  computerized  evaluation  and  negotiation  the 
^ standard  technique  In  five  — ten  years. 

Ocher  computer  advances  will  be  Incorporated  In  the  evaluation 
and  negotiation  application.  The  conputer  angnented  with  a 
' _ printout  capablUty  provides  another  dlnenslon.  It  la  expected 
that  In  a stalenate,  the  selected  lower  level  details  at  issue 
will  be  few  in  nunber  and  can  be  selected,  printed  out,  and 
nore  thoroughly  pursued  by  engineers,  accountants,  and  other 
analysts  representing  the  parties  Involved. 
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“ proceed  with  co^mterleed  eppUcetlon 
la  a coordinated  effort  with  Servlce/DOD  and  contractor 
pemonnel.  • One  progran  and  one  data  baae  la  all  tK«f 
ne^a^  for  each  najor  contractor.  Thla  coamm  naage  will 
not  only  be  efficient  In  the  devcloplag  of  the  ayatcna.  bat 
offera  adwantagea.  A alngle  program  and  a olngle 

•oarce  of  data  wonld  alao  permit  a aignlf  l^t  amoaS^  the 

conducted  over  telephone  Unea  with  both 
partlea  hawing  dlaplay  unite.  The  day  ahould 
preliminary  negotlatlona  wlU  be  conducted  by  phone  and 

**^‘*‘®  P“tlea  hawing  wery  little  contact  with 
each  other,  tbna  mlnlmlalng  the  coat  of  trawel  and  loan  of 
peraonnel  for  other  dutlea  while  on  trawel  atatua.  Hben 
ewalnatlm  flndlnga  are  dcweloped,  thin  Information  la  fed 
“ • ■ •>  the  coMuu  computer  program  by  the 

nodltor,  price  analyet,  or  Induatrlal  engineer.  A nfanle 
notice  to  the  negotiator  of  the  quantified  entree  with 
•tatcmenta  aa  to  why  will  replace  the  preaent  wolunes  of 
poorly  cownlcatlng  propoaala  and  reporta.  ...j 

It  will  be  Intereatlng  to  ace  esactly  what  »■«— » factor 
chaagea  occur  in  the  Indlwldnal  negotiating  partlea  aa  a 
re^t  of  thla  Improwement  In  the  computer  application.  In 
r**  new  planned  program,  an  la  other  computer  appUcatlona, 

It  wmld  seem  the  computer  Itaelf  la  a participant,  along  with 
both  negotlatore,  aa  It  alto  patiently  waiting  for  a human 
rej^ion  to  ita  laat  Inatant  adjuetacnt  of  waluea.  • It  la 
o^ted  that  the  computer  would  drlwe  the  two  partlea  In 
the  bargain  to  an  earlier  resolution  of  their  differences. 

i 

In  applications  other  than  contract  negotiation,  Mjor 
souRe  selection  and  program  approvals  could  be  analyzed  and 
evaluated  via  these  terminals  with  the  same  safeguards 
n*®*****7  to  prevent  Industrial  espionage  via  computer. 

Cost  Information  can  then  be  made  available  to  higher  echelons. 


o 

U- 

o 
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■ 
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This  exploitation  of  computer  science  application  to 
^i^e  our  archaic  proposal,  evaluation  and  negotiation 
business  methods  is  Important  as  cost  becomes  more  and  more 
an  Issue  In  major  system  decisions.  We  cannot  exist  In  an 
era  where  cost  Is  an  equal  partner  with  performance  and 
schedule  wlthmit  Improving  our  capability  to  manage  cost 
Information. 


159 


OVERCOMIIiC  ISEKTIA 


u 

u 

O 

O 

O 


Wille  all  tba  foreaeeable  and  Innovatlva  oaea  will  not 
■atariallza  coapletely  in  cbe  naxt  five  - ten  yaara.  thara- 
la  no  doubt  that  thara  will  ba  algnlflcant  prograaa  along 
thaaa  roada  during  that  tine  parlod.  Tha  only  datarrant  In 
—king  prograaa  will  Ua  In  apathy  and  raalatanca  to  change 
aaaoelated  with  tha  conaarvatlan  of  tha  coat  analyala  and 
''  contracting  rnununlty.  If  a few  contractor  and  Govemnent 
nanagara,  dedicated  to  tha  laprovanent  In  procnraaent  through 
raaaarch,  and  othera  haring  a nora  paaalng  Intereat,  will 
place  eaphaala  on  conpoterlzad  tachnlquaa,  Inprovcnent  of 
najor  weapon  ayatcaa  coating  will  cone  to  fruition.  Aecura47 
and  tlnellnaaa  of  coat  Infomatlon,  both  of  which  can  ba 
Inprorad  through  conputer  appllcatlona,  are  abaolnte 
nacaaaltlaa  If  coat  la  Intended  to  ba  an  equal  partner  In 
— Jor  weapon  ayaten  daelalona.  The  tine  for  coat  profeaalonala 
to  change  bualnaaa  practice  la  here,  Juat  aa  tha  *•«—  haa 
apparently  cone  for  the  legal  profaaaion.^ 
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Vhco  the  task  la  at  a ■aetlng  place  vith  technology, 
the  inertia  haa  been  'overcone  # £f forte  vere  made  in  certain 
Any  source  aelectiona  to  conputerixe  aa  such  of  the  coat 
infomatioQ  aa  uaa  practicable.  The  aignif leant  difference 
in  the  aK>re  recent  Amy  efforta  to  nore  fully  utilize  conputera 
liea  in  the  attenpt  to  not  only  conputerize  the  contractor'a 
propoaal  in  the  hiatoric  elenenta  of  coat,  i.e.,  mterial, 
■anufacturing,'  nanhoura,  overhead,  but  to  alao  conputerize 
the  propoaed  coat  by  lover  levela  of  the  work  breakdovn 
atructore.  The  next  portion  of  thia  preaentation  will 
diacuaa  the  lover  level  data  laaue  aa  a factor  in  Inproving 
coat  eatinating  and  evaluation. 


TAILOR  EXISTIHG  SYSTEM 

The  concept  for  C/SCSC  aurveiUance  for  ooau  control  during 
a prograng  aa  Identified  in  the  Departnent  of  Defenae  guide, 
indicatea  that  aurveiUance  beglna  vith  contract  award. ^ 


^Jack  Bemateln,  Conputera  Make  Lavyera*  Work  Eaaier.  Cooqmtera 
and  People  (May  1977)  P.  25. 

^C/SCSC  Joint  Surveillance  Guide.  Amy  Materiel  Coenand  Paiq>hlet, 
AMCP  715-10,  et  al,  1 July  1974,  P.  1-2. 
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Presentlyt  ^be  contractor  lays  out  hia  discrete  budget 
portions,  aJter  contract  award,  and  has  to  pass  a review 
and  systea  desamst ration  under  the  C/SCSC  systca.  This  base 
line  is  uncoordinated  and  unreconciled,  except  for  the 
contract  line  itea  control,  with  the  basic  estiaates  of  the 
contractor  that  were  developed  during  either  a source  selection 
process  or  during  the  contract  negotiation  process  • The 
reason  for  the  absence  of  coordination  and  reconciliation  in 
the  basic  budget  as  required  by  the  C/SCSC  systea  is  generally  the 
VBS,  which  is  not  present  because  of  the  current  practices 
in  proposal  preparation,  evaluation,  and  negotiation*  The 
following  two  tjd>les  show  cost  as  siaBarixed  in  proposals 
(Table  3)  and  as  sunaarized  in  the  C/SCSC  systea  UBS  (Table  4). 

f;  1.  .‘if  r;«  * * ^2,1^  ■*2'  liVi' 

, TABlJt  3 ^ 

TYPICAL  PROPOSAL  SUHMAET  ,4 

(VALDES  IH  THOUSANDS) 


COST  ELEMENT 

. . _ PROPOSED 

Materials 

■ V - /*.  " $10,008 

Matl  Burden 

......  1.112 

TOTAL  MATL 

• . . $11,120 

Engr  Hrs 

, J ' (57) 

Engr  Dollars 

855 

Engr  Overhead 

1.283 

Mfg  Hrs 

. (375) 

Mfg  Dollars 

2,994 

Mfg  Overhead 

5,988 

Other  Cost 

5.562 

TOTAL  MFC  COST 

$27,802 

GAA 

4.704 

TOTAL 

$32,506 

Profit 

2.994 

TOTAL  PRICE 

$35,500 

TABLE  4 
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HBS  CODE 


TTPICAL  C/SCSC  SUIMART 
(VALDES  IH  THOOSANDS) 


• 


ELEMENT  IDENTinCATION 


BASELINE  DOLLARS 


1.1 

- Air  Vehicle 

35,500 

1.1.1 

AlrfroM 

^ ■ 28,621 

l.l.l.l. 

Fhselage 

‘ " ' 7,008 

1 

- ' - ' - Mom  SmcIod 

1,142 

2 

' Mid  Section 

' 2,855 

3 

Aft  Section 

1,194 

4 

Main  Pylon 

368 

5 

TaU  Cone 

347 

6 

TaU  Pylon 

586 

7 

Stabilizer 

440 

8 

Integration 

76 

1.1.1.2 

Landing  Gear 

373 

Main  iiSndfng  Gear 

228 

Tall  landing  Gear 

145 

I.I.I.3. 

Transmission 

3,304 

1 

Main  Gear  Box 

2,657 

2 

Intermediate 

68 

3 

TaU 

189 

4 

Shaf  tlxig 

345 

5 

Other 

45 

1.1.4 

Armament 

71 

1.1.5 

Integration 

4,824 

NOTE;  Shown  are  only  22  of  over  60  elenents  thru  the  5th  level. 


lf»  hovever^  In  a new  frontier  of  coat  estimating,  ve  would 
develop,  at  the  time  of  proposal  preparation,  the  same  lower 
level  of  detail  that  would  be  ultimately  utilized  for  contract 
coat  control  during  the  program's  operation,  we  %rlll  have 
started  the  surveillance  at  Its  true  beginning.  The  contractors 
must  prepare  a lower  level  detail  estimate  which  continues 


to  utilize  the  eleaents  of  cost,  such  ss  labor,  ■sterlels, 
and  burdens,  Mit  Bust  also  coaplete  the  aatrlz  to  include 
lower  level  of  detail  cost  at  the  Sth,  6tb,  and  7th  levels 
of  the  VBS.  Results  of  this  type  of  proposal,  evaluation, 

, and  negotiation  activity  will  form  the  base  line  used  by 
^ the  contractor  In  the  C/SCSC  dewonstratlcw.  This 
; that  cost  account  aanagers  participate  folly  In  developing 
' the  estloate  which  becooes  the  budget  and  which  la  loaded 
into  the  contractor's  cowputer  systew  before  the  award  la 
) Bade.  Table  5 la  an  ezaaple  of  the  new  UBS  coaqiuter  data. 
Ihe  fifth  level  aaln  gearbox,  the  largest  hardware  IteB  in  ■ 
the  transBlsslon  WBS  as  shown  In  Table  4,  la  shown  In 
detail  In  Table  5. 


TABLE  5 


MAIM  GEARBOX  \ 

(TALOES  IB  THOPSAuDS)  ' 


VBS 

LEVEL  5 

COST 

1977 

1978 

1979 

TOTAL 

ELEHENT 

CONTR 

GOVT 

COHTR 

OOVT 

CONTR 

OOVT 

CONTR 

GOVT 

Materials 

410 

390 

326 

310 

42 

39 

778 

739 

Matl  Burden 

28 

28 

23 

22 

3 

3 

54 

53 

TOTAL  MATL 

438 

418 

349 

332 

45 

42 

a32 

792 

Eng  Hra 

(4.0) 

(3.3) 

(3.3) 

(3.3) 

(3.3) 

(3.3) 

(10.6) 

(9.9) 

Eng  Dollars 

24 

20 

20 

20 

20 

20 

64 

60 

Eng  Overhead 

36 

30 

30 

30 

30 

30 

96 

90 

Mfg  Hrs 

(7.2) 

(7.2) 

(17.8) (17.1) 

(7.0) 

(6.1) 

(32.0) (30.4) 

Mfg  Dollars 

50 

50 

125 

120 

49 

43 

224 

213 

Mfg  Overhead 

100 

100 

250 

240 

98 

86 

448 

426 

Other  Cost 

93 

93 

232 

223 

91 

80 

416 

396 

TOTAL  COST 

741 

711 

1006 

965 

333 

301 

2080 

1977 

GAA 

TOTAL 

126 

867 

121 

832 

171 

U77 

165 

1130 

57 

390 

50 

351 

354 

2434 

336 

2313 

Profit 

80 

77 

108 

104 

35 

33 

223 

214 

TOTAL  PRICE 

947 

909 

1285 

1234 

425 

384 

2657 

2527 

! 

v; 

O 
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Thus,  a coQtinuity  la  developed  between  the  values  of  the 
contractor  proposal » as  adjusted  by  the  negotiation  process • 
and  the  program*  s base  line  which  will  be  used  by  the  contractor 
during  contract  performance.  For  those  a little  more  familiar 


with  the  C/SCSC  system  and  CPR  reporting,  the  only  major 
reconciliation  item  between  the  base  line  for  program  coot 
control  during  contractor  pexfoimance  and  the  final  evaluation 


and  negotiation  amount  will  normally  be  the  contractor's 
management  reserve.  Any  other  difference  would  make  it 


inembent  on  the  contractor  to  identify  and  explain  fully. 
By  implementing  the  lower  level  WBS  proposal,  analysis,  and 
negotiation  concept,  there  should  be  an  improvement  in  the 
base  over  that  developed  in  the  preseuc  postcontract 


period 


EARLIER  AND  BETTER 


) In  looking  at  the  implementation  of  C/SCSC  system  validatioas, 

the  cost  performance  reports  (CPR)  and  uniform  VBSs,  we  have 
trained  contractor  and  Government  personnel  to  understand  and 
accept,  that  for  proper  cost  control,  budgets  must  be  formulated 
at  lowest  level.  Ve  see  that  individuals  are  assigned 
responsibilities  for  those  discrete  portions  of  the  budget 
• that  are,  in  fact,  manageable  within  the  human  limitations  of 
time  skill.  The  examination  for  use  of  lower  level 
detail  in  major  weapon  system  costing  is  nothing  more  than 
to  advance  the  C/SCSC  application  and  to  develop  this  information, 
not  in  the  postcontract  period,  but  as  a part  of  the  original 
proposal  preparation.  It  is  believed  that  in  this  period  of 
time  the  contractor's  best  effort  would  be  put  forward  and 
I hla  best  estimates  generated.  It  is  those  best  estimates 

that  should  beceme  the  common  base  line  for  use  by  the 
contractor  and  the  Government  in  all  future  activities. 

Having  examined  computer  science  factors,  the  relationship 
to  C/SCSC  system  and  uniform  WBS,  it  is  appropriate  to  discuss 
the  source  and  problem  of  data  availability.  If  the  contractor 
develops  estimates  at  the  lower  levels  earlier,  and  the  computer 
accepts  all  that  data,  how  are  ve  going  to  evaluate  and 
negotiate? 
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LOWER  LEVEL  DATA 


At  this  point  in  the  evolutionary  process  of  improving 
estimating  accuracy  through  the  use  of  nev  estimating 
techniques,  it  is  necessary  that  collaborating  and  supporting 
<lata  be  made  available  for  evaluating  the  contractor's 
proposal.  This  data  must  also  be  at  the  lower  level  of 
detail  to  adequately  satisfy  this  need.  lamedlately  this 
area  would  be  seized  upon  as  being  a prohibited  deficiency 
requiring  contractor  and  Government  manpower  beyond  th^ 
capabilities  of  skill  and  time,  thus  rh^  use  of 

lower  level  detail  for  decision  contracting 

purposes.  However,  on  major  weapon  systems  this  simply 
cannot  be  the  case.  If  contractors  are  truly  pursuing 
design- to-cost  and  are  considering,  in  their  management 
decisions,  that  cost  is  an  equal  partner  with  other  require- 
ments, then  this  lower  level  of  detail  most  be  available. 

A recent  article  on  design— to— cost  is  a good  example  of 
lower  level  of  design- to-cost  efforts*^ 

If  the  data  is  not  available  to  the  contractor  and  thus 
available  to  the  Government^  the  contractor  is  not  performing 
his  design-to-cost  requirement  by  permitting  cost  to  be  a 
consideration,  and  in  fact  an  influence,  on  the  design  of 
lower  level  component  hardware.  Even  if  prototyping  or 
the  prototyping  schedule  dictates  that  higher  cost  components 
are  initially  designed,  this  design  most  be  recognized  by 
the  contractor,  in  doing  design— to-cost  gnd  —yir-fyig  cost/ 
performance  trade-off  decisions,  as  requiring  change  for 
the  production  configuration.  One  of  the  outputs  of  an. 
active  and  dedicated  design— to— cost  effort  will  Include 
the  tracking  of  original  estimates  anc*  the  actual  cost  of 
these  items  for  all  of  the  a^djor  cost  drivers  in  a major 
weapon  system.  When  doing  either  competitive  or  sole 
source  prototyping,  the  cost  community,  including  both 
the  contractor's  and  Government's  participants,  must  take 


^COL  Ronald  E.  Philip  and  William  P.  Wood,  Design-to-Cost 
Effective  in  Artillery  Program.  Government  Executive  (April  1977), 
P.  32. 
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advantage  of  the  avallabilitj  of  actual  coats  incurred  on  * " 

loirer  level  cooponenta  as  part  of  the  Banufacturlng  process 
for  the  prototype.  One  saving  consideration  la  that  there 
la  a United  number  of  cost  drivers  at  lover  levels.  For 
example  • 80  - 85Z  of  the  utility  helicopter  manufacturing 
costs  are  In  the  follovlng  UBS  Items: 

t .•  .).  • .vji  c’f.  -r  ' ’ u^.  r xr-:*  ' 

: / . table  6 ’ ^ 

k.  ^ - A rt'r":  : ^ .*-■  *.5^  v.-r 

' MAJOR  AIRCRAFT  COST  DRIVERS  * ' 


Hose  Section 

Servos 

Nod  Fuselage  Section 

APU 

Aft  Fhselage  (Transition) 

Electrical  Power  System 

Main  Rotor  Pylon 

Electrical  Harnesses 

Tailcone 

Instruments /Displays 

Tall  Rotor  Pylon 

Cabln/Cockplt  Planishing 

Stabilizer 

Engine  Installation 

Main  Gear  Box 

Tall  Rotor 

Main  Rotor  Blades 

Main  Landing  Gear 

Main  Rotor  Head 

Tall  Imnding  Gear 

Mechanical  Flight  Control  Sys 
Hydraulic  Flight  Control  Sys 

Integration 

It  Is  recognised 9 of  ccorse,  t^at  two  adjustments  nay  be 
necessary.  First,  the  prototype's  design  nay  need  to  be 
changed  for  production  because  of  either  the  time  constraints 

Identified  before  or  because  later  rAurtng  reveals  the  components  i 

will  not  meet  other  requirements.  Second,  since  the  prototype  1 

Is  generally  a one  of  a kind  nanufacturlng/purchase.  adjustment 

may  have  to  be  made  to  the  material,  engineering,  and  manufacturing 

hours  for  the  production  mode.  These  adjustments  would  be 

for  more  sophisticated  tooling,  volume  buying,  optimizing 

the  utilization  of  plant  facilities,  etc. 

COLLECT  DATA 

It  la  Important  to  point  out  that  all  through  major  weapon 
system  development,  the  technical  community  has  always 
availed  Itself  of  the  opportunity  to  change  design  (critical 
design  review),  monitor  the  test  results  (including  actual 
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witnessing  of  the  test),  end  coerce  the  contractor  to  add 
tests  or  speciaens  when  results  are  Inconclusive.  Thus, 
engineering.  In  assuring  contractors  nect  nalntalnabllity, 
reliability,  hunan  factors,  structure,  fatigue,  and  a whole 
litany  of  other  technical  requlrenents  has  augnented  the 
contractor's  technical  staff  with  their  own  experts  and 
d aanded  lower  level  detail  In  support  of  conclusions.  The 
cost  cn—mlty  hea  always  deaanded  aom  suaaMrlzed  cost  data 
after  the  fact*  If  we  arc  truly  to  aake  cost  an  e<iaal 
partner,  we  aust  aake  the  cost  estlaate  of  coaparable  quality 
so  that  In  the  decision  asking  process  it  can  truly  be  an 
equal  consideration.  Lower  level  cost  data  collected 
selectively,  on  a real  tlae  basis,  is  a requlreaent  needed 
to  execute  expanded  cost  responsibilities. 

EQOAIK  lESOOaCES  . ..  f 

To  aake  cost  an  equal  partner  will  require  the  sane 
resources  of  tlae,  people  sad  aanageaent  inforaatlon  systeas 
long  used  by  the  technical  cnniilty  la  developing 
evaluation  of  the  contractor's  hardware.  The  point  being 
aade  Is  that  we  cannoc  expect  that  cost  Is  an  equal  partner 
until  we  are  ready  to  eatabl  tsh  laproved  cost  data  systeas 
and  to  staff  the  cost  evaluation  organisation  with  the  nuaber 
eed  skill  level  necessary  to  laprove  cost  accuracy  for  decision 
purposes. 

1 

Thus  far,  the  engineers  and  operations  research  analysts, 
who  In  their  careers  are  specializing  la  aajor  weapon  systea 
costing,  have  seen  In  this  paper  areas  of  significant  Interest. 

In  the  new  frontier  aore  Industrial  engineers  skilled  In 
specialized  areas  associated  with  the  developaenu  and  aanufacturlng 
of  lower  level  coaponent  hardware  will  be  require.  More 
specific  aanufacturlng  knouledge  In  the  area  of  cooponents 
not  heretofore  evaluated  In  detail  will  be  an  absolute  require- 
aent  and  necessitate  an  enhanceaent  In  our  technical  evaluation 
skill.  This  Issue  of  note  and  laproved  costing  gUn  Is 
vitally  laportant  If  costs  are  to  truly  be  an  equal  partner 
In  decisions  related  to  major  weapon  systems.  The  operations 
research  analysts  will  utilize  the  actual  component  cost 
data  to  improve  estimates  of  future  aircraft  or  aodlflcptloos 
to  existing  aircraft.  The  opportunity,  through  regression  analysis 
os  lower  level  detailed  data,  to  develop  a cost  per  pound 
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estlBatlng  capability  is  of  aigoificant  intereot*  He  sees 
the  oev  frontier  in  cost  evaluations  and  analysis  as  not 
being  Just  more  of  the  same»  hut  %iill  he  an  opportunity 
for  innovative  and  inventive  research  with  the  new  pover 
level  data.  In  one  Army  production  contract,  lover  level 
production  (not  prototype)  actuals  are  being  collected  by 
production  lots  in  a continuing  effort  to  advance  research 
Ir.  this  cost  estimating  methodology.^ 


Those  readers  vho  are  in  the  field  and  vho  have  mathematical 
or  scientific  degrees  or  backgrounds  have  been  identified 
vith  this  forecast  of  the  next  five  • ten  years.  But  the 
lovly  accountant  has  been  patiently  vaiting  and  has  seen 
finriiing  for  him.  But  the  accountant  and  his  fellov  business 
mgjors  mill  have  a place,  a very  important  place  in  this 
new  frontier.  His  place  is  assured  by  the  presence  of 
indirect  cost.  The  indirect  cost  can  only  he  evaluated 
and  negotiated  vith  the  skills  of  the  price  analyst,  economist; 
business  analyst,  accountant,  and  anditor. 


ETAUIATE  INDIRECT  COST 


For  the  most  part,  the  discussions  of  the  new  frontier 
have  dealt  %rith  direct  cost.  But  if  data  by  lower  level 
VBS  and  computer  science  is  to  improve  cost  estimating, 
the  accuracy  of  indirect  cost  must  be  improved  concurrently. 
If  Indirect  costs  are  not  addressed  to  i^rove  their  quality, 
then  the  total  cost,  even  at  the  lower  levels  of  the  VBS. 
cannot  be  expected  to  significantly  improve. 


The  Pie  Chart  shows  or  depicts  a generalized  multi-million 
dollar  airframe  contract  which  segregates  the  direct  type 
costs  normally  addressed  in  an  evaluation  and  negotiation 
situation  from  the  indirect  costs  which  are  concentrated  on  more 


;L‘  

^CCDR  Plan  Attachment  113  and  Exhibit  A.  Addendum  to  DI-F-6008. 
/ . PM-bOA  Utility  Transport  Aircraft  System.  Sikorsky  Aircraft 

• and  US  Government  (DAAJ01-77-C-0001) . P.  51. 


by  the  DCAA  auditors  and  the  ACO  than  by  the  Source  Selection 
Evaluation  or  the  Should  Cost  Team.  The  chart  che  ' 

liqtortanc  point  that  this  Indirect  cost  is  at  least  as  great  . 
as  the  direct  cost  and  thus  requires  the  sane  degree  of  attention. 
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' In  order  to  assure  that  we  are  able  to  buy  the  best 
systea  that  our  budgets  can  afford.  In  this  new  frontier, 

■ich  Bore  effort  must  be  put  on  the  proposal,  evaluation, 
and  negotiation  of  the  Indirect  cost.  It  Is  significant  to 
point  out  that  with  varying  degrees  of  Intensity,  Indirect 
costs  are  being  managed  as  a part  of  the  audit  and  accounting 
functions  during  contract  perfomance.  It  Is  only  within 

years  that  the  Air  Force,  Navy  atwi  Amy  have  attenpted 
varying  degrees  of  effort  In  addressing  the  Indirect  costs 
through  overhead  nonltorshlp.  These  efforts  have  been  directed 
at  cost  control  during  contract  perfomance. 


What  has  not  been  sufficiently  eapbaslzed  Is  the  thorough 
and  detailed  evaluation  of  Indirect  coat  as  part  of  t-*»* 
program  estimate,  contract  estimate,  or  source  selection 
otlmate.  A thorough  examination  of  the  pools  op 

Indirect  cost  and  the  base  making  up  the  distribution  of 
that  cost  must  be  made  for  each  and  every  known  contractual 
participant  included  In  a program.  There  Is,  a need  to 
establish  continuing  responsibilities  and  reconciliations  » 
for  cost  charged  direct,  but  estimated  thru  some  allocation  • 
or  factoring  method.  Beal  time  Indirect  cost  Information 
Is  an  absolute  necessity  In  order  to  be  In  a position  to 
have  confidence  In  the  cost  Information  sufficient  to  permit 
It  (the  cost)  to  be  a competing  equal  partner  with  the 
offerings  of  the  technical  cfiiilty.  This  paper  does  not 
propose  to  address  the  detail  changes  necessary  to  Improve 
the  accuracy  of  Indirect  cost.  That  topic  la  worthy  of  a 
separate  paper  dealing  only  with  indirect  estimating, 
evaluating  and  control. 

Is  the  new  frontier  going  to  be  Involved  more  and  more 
in  the  contractor's  business  throwfjh  a rltrvrr  look  at 
accounting  atructure  which  Is  gencratlag  t-t«e  Indirect 
cost?  It  must  be.  If  half  of  the  cost  accumulated  by  the 
major  weapon  system  is  from  these  Indirect  accounts. 
we  have  no  less  a responsibility  to  thoroughly  pursue  rh*se 
business  costs  chan  the  responsibility  we  shoulder  when  we 
evaluate  and  negotiate  a bill  of  material  cost  and  the 
engineering  and  manufacturing  labor  cost.  These  business 
costs  are  so  called  In  this  paper  becanae  only  the  business 
*®J®cs.  the  accountant . the  auditor,  the  financial  analyst, 
and  the  economist  offer  the  skills  necessary  to  perform 
the  thorough  review  and  arrive  at  more  accurate  estimates 
In  evaluation  and  negotiation  of  Indirect  costs.  If  contractors 
do  not  establish  systems  to  better  assign  responsibilities 
and  accountabilities  for  business  cost,  the  new  frontier 
will  see  more  and  more  Government  analysts  Involved  In  more 
detailed  evaluation  and  analysis  of  projected  Indirect  coat. 
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OPERATING  ANO  SUPPORT  (OAS)  COST 

By  capitalizing  on  cbe  aivallablllty  of  aore  accurate  lower 
level  coat  data,  ays tens  entering  production  can  be  selected 
based  on  Inproved  04S  cost  Infonutlon.  Tbls  new  methodology 
attacks  the  weakness  In  the  accuracy  of  hardware  cost, 
thus  It  laproves  that  portion  of  O&S  cost  estlnates  that  can 
be  Identified  for  i]q>roveunt  by  the  cost  analysis  coawmlcy. 
Host  of  the  other  sbortcoalngs  In  cost  accuracy  are  traceable 
to  the  optlalsa  of  cbe  contractor  and  the  Covemaent  In 
assessing  technical  requlrcaents.  For  a long  tlw  the  cost 
analyst  has  taken  the  brunt  of  crlclclsa  for  cost  growth, 
when  perforsunce  growth  was  the  heart  of  the  problen. 
Beplenlshaent  spares  are  the  portion  of  OAS  cost  which  can 
be  laproved  In  cost  accuracy  through  the  use  of  lower  level 
detail  estlstates  and  evaluation  via  the  cosqputer.  Although 
the  replenishaent  spares  are  only  16Z  of  the  OAS  cost  of  a 
utility  helicopter,  they  are  one  of  Che  large  cost 
ItCBS  and  are  possibly  the  most  discriminatory  between 
cowpetlng  systems  or  configurations  within  a define  specifi- 
cation. The  balance  of  the  cost  Is  largely  direct 
Indirect  people  cost,  and  except  for  direct  maintenance, 
^c*iet  discrimination  between  systems  competing  for  the 
same  requirement.  The  lower  level  detail  improves  cbe 
accuracy  of  the  hardware  portion  of  OAS  cost  since  it 
Improves  the  cost  estimate  of  new  replacement  parts,  repair 
of  parts  and  overhaul  of  parts.  Therefore,  computerized 
UBS  lower  level  data  estimates  will  Improve  the  OAS  as  well 
as  Che  production  cost  estimate. 

OmPONEKT  DOLLAS/POUND  NOW 

Systems  entering  development  can  benefit  from  the  new 
methodology.  Noncontract  estimates  for  future  systems 
or  modification  to  existing  systems  can  be  developed  using 
operations  r< search  techniques.  By  capturing  cost  at  a 
lower  level  of  detail.  It  becomes  possible  to  compare  cost 
and  some  related  forecasting  parameters  through  the  use  of 
regression  analysis.  With  several  systems  collecting 
costs  and  weight,  the  next  five  to  ten  years  would  permit 
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the  developnent  of  cost  per  pound  estlastlng  at  the  contpooent 
level.  Cost  per  pound  has  been  identified  onlf  because  weight, 
at  least  in  the  forecasting  of  cost  for  helicopters,  been 
the  single  aost  reliable  paraMter.^  This  would  be  a valuable 
tool  to  use  as  older  systev  arc  considered  for  ■odiflcation 
involving  SOM  redesign.  Cost  per  pound  forecasting  of  that 
functional  coaq>onent  would  be  still  another  estlastlng  tool 
within  the  repertoire  of  the  cose  analyst  for  use  in  tae 
decision  asking  process.  Cnrrently,  the  OTTAS  prograa  la 
collecting  both  the  cost  and  correspondence  weights  of  soae 
selected  ccaponents  as  a first  step  la  establishing  a data 
base  at  the  coaponent  level. 

■ • . c J-  ....  JU 


BENEFITS 


The  benefits  fron  this  evolving  technique  esn  be  stMsrized 
in  seversl  specific  categories  associated  with  najor  ifeapon 
•y*^«*«*  First,  the  quality  of  the  estimate  for  the  Instant 
and  any  options  ia  improved  over  historic  methods, 
^le  Increased  quaUty  is  not  an  Inherent  value  of  lower  level 
where  both^  contractor  and  Govemaent  technical  people 
ere  aore  presonally  knowledgeable  of  the  hardware 
the  opportunity  to  develop  an  accurate  cost.  As  the  proposal 
is  evaluated  and  subsequently  negotiated,  zitm  quality  of  the 
®®fl*ste  is  laproved  as  the  interplay  of  rh*  experts  is 
^ftoduced  and  resolutions  and  conclusions  are  developed  with 
respect  to  differences. 


Second,  once  the  contract  is  signed,  this  detailed  docnaenta- 
tloQ  beccaes  the  detailed  base  line  for  the  contract  value. 
Changes  to  the  contract,  and  particularly  dianges  to  the  hardware 
becoM  easier  to  estiaate  accuratelv.  gy  isolating  an  itM 
sudi  as  the  aaln  gear  box,  the  change  to  that  coaponent's  gear 
can  be  aore  accurately  costed.  Both  the  contractor  and  the 
Govemaent  are  better  able  to  Isolate  the  pool  of  costs  involved 
in  s change  and  evaluate  and  negotiate  the  adjustawnt  to  that 
discrete  pool  of  cost.  Both  the  contractor  and  the  GoveroMot 
should  be  in  a better  position  to  evaluate  engineering  changes 
to  hardware,  both  before  they  are  contractually  approved  and 
when  they  becesM  a bilateral  adjustment  in  settlement  of  a 
contract  change  order. 


^Stanard.  G.  Norman,  Parametric  Life  Cycle  Cost  Model  for  Amy 
Helicopters.  Proceedings  Tenth  Annual  Department  of  Defense  Cost 
Analysis  Symposium,  hosted  by  the  Comptroller  of  the  Army,  Alrlie 
House  Conference  Center,  Alrlie,  Virginia,  p.  72. 


The  third  area  even  precedes  activities  associated  with  a 
coatractual  change.  This  is  the  area  wherein  the  Covernnent 
a decisloo  based  on  performances  or  logistics  versus 
cost.  In  reviewing  a hardware  change » accurate  cost  data, 
available  at  this  point  in  timt^  will  permit  cost  considerations 
to  be  an  equal  partner  with  other  parameters.  As  cost  becomes 
a more  and  more  Important  consideration  in  the  decision  process, 
accurate  costs  must  be  present  at  more  and  more  locations  along 
the  path  of  both  contractual  aai  program  progress.  When  require- 
ments are  considered  to  be  changed  by  engineering,  product  ^ 

assurance  or  tb^  user,  cost  estimates  of  quality  must  be 
iMediately  available  for  the  cost  trade-off  decision.  With 
the  availability  of  cost  data  will  come  an  awareness  of  cost 
consideration  on  the  part  of  the  technical  cooBunity.  Cost  can 
become  part  of  tbiA  trade-off  analysis  rather  than  be  treated 
as  an  adversary. 

ACCEPTANCE,  COQBOINAXION  AND  RESOUBCES 

Ubat  are  the  current  limitations  preventing  full  implementation 
of  rbia  lower  level  estimating  technique?  The  limitations  lie 
in  only  three  areas.  First,  more  and  more  contractors  mist  be 
encouraged  to  accept  the  innovative  technique  and  develop 
computerized  programs  peculiar  to  their  accounting  and  estimating 
systems.  They  must  familiarize  their  personnel  with  the 
financial  policy  that  estimates  «rtll  be  generated  at  the  lover 
Xevel  of  detail  and  such  estimates  will  be  fed,  in  some  suitable 
format,  into  the  computer  for  higher  level  summation.  They 
must  recognize  Government  access  to  that  data  on  a real  time 
basis  and  be  prepared  to  consider  Government  evaluating  input 
data  to  their  programs  on  a concurrent  real  time  basis. 

Coordination  is  the  second  area  thwarting  accelerated  application 
of  the  technique.  Coordination  will  be  required  more  for  the 
Government  aivi  contractor  negotiators  and  financial  managers 
than  any  other  group  involved  in  the  cost  estimating  and  analysis. 
Contractor  estimators  are  already  somewhat  familiar  %rith  the 
development  of  estimates  by  components;  however,  often  these 
estimates  have  lost  their  individual  identify  in  the  final 
proposal  submission.  Similarly,  the  Government  engineers  and 
the  audit  position  are  often  obscured  in  the  Government  evaluation 
position.  The  best  estimate  must  be  entered;  or  if  adjustments 
are  made,  they  must  be  with  full  knowledge  and  understanding 
of  the  estimators  and  evaluators. 


Finally,  both  the  contractor  and  the  Government  must  identify 
sufficient  people  resources  and  raise  the  skill  level  of  these 
people  to  permit  cost  accuracy  that  is  in  keeping  with  its 
present  equal  status  to  performsnce  and  schedules.  The  days  %ifaen 
organizations  operate  with  hundreds  of  engineers  and  a fev  cost 
practitioners  should  not  he  permitted  to  continue  in  the  new 
frontier.  The  smse  imbalance  mixture  of  professional  personnel 
mill  not  allow  the  new  frontier  to  achieve  any  significant 


improvement  even  if  we  try  superhumanly  to  make  cost  an  equal 


partner. 


Not  most  the  allocation  of  resources  he  reexamined, 
bat  the  cost  estimating  and  control  responsibilities  should 
be  reviewed  for  the  appropriateness  of  the  organizational 


location.  Presently  major  weapon  system  cost  responsibilities 


are  often  part  of  the  contractor  and  Government  coatptroUer 
function. 


Vitboot  going  into  great  depth  since  each  organization 
would  need  to  be  reexamined  based  on  its  own  structure,  it 
can  be  generally  said  to  be  mostly  involved  with  accounting 
and  finance  and  their  related  decisions  and  reports.  Vithin 
that  organization  the  major  weapon  system  cost  responsibility 
is  a minority  in  the  total  manpower  picture. 

The  Government  at  the  **doer**  level  generally  can  be  stated 
as  having  the  exact  same  situation.  The  comptroller  function 
at  major  subordinate  commands  devotes  the  majority  of  resources  to 
other  than  major  weapon  system  estimates.  Vithin  that  organiza- 
tion the  major  weapon  system  cost  responsibility  is  a minority 
in  the  total  manpower. 

One  solution,  which  is  a mere  application  of  the  C/SCSC 
**Earlier  and  Better"  is  to  centralize  the  major  %reapon  system 
cost  responsibility  in  the  procurement  organization  where  the 
significant  spending  of  program  funds  starts  and  finishes  as  part 
of  the  contractual  arrangesients.  The  new  frontier  should  consider 
assigning  program  cost  control  responsibilities  from  the  time  of 
initial  estimate  at  the  start  of  the  life  cycle  of  major  system 
acquisitions  to  the  procurement  executive  in  both  the  contractor 
and  Government  organization.  The  procurement  contracts  are  the 
source  of  cost  data  and  the  production  skills  are  the  sanitizers 
of  that  data. 


This  view  of  the  rew  frontier  offers  no  compelling  evidence 


that  there  is  a narrow  path  to  follow,  but  envisions  there  is 
a broad  avenue  for  progress. 
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ttmmiM.  OPPORTUKITIES  m LlMTimttlS  H»  TOE  USE  Of  WKMIBflff  IBailOt 


Or.  Paul  F.  Arvls 

Director,  US  Any  Procurement  Research  Office  -it?  < 

Fort  Lee.  YA  i . 


RELATIONSHIP  OF  SUBJEa  TO  THEWE  • ' * 

The  translation  process  as  described  by  General  Sharp  Is  primarily  a function  of  mnage- 
nent.  supported  by  research  and  operational  elements.  For  the  next  few  minutes  Td  like 
to  suggest  some  opportunities  for  performing  this  function  and  examine  some  limitations  or 
barriers  we  need  to  be  on  guard  for.  I will  cite  actual  experience  where  appropriate. 

CHANGING  SOURCE  OF  RESEARCH  PROJEa  GENERATION 

Up  until  not  too  many  years  ago,  miay  research  projects  In  govenmental  and  Industrial 
laboratories,  think  tanks,  and  Institutes  were  generated  by  the  researchers  themselves. 
Research  projects  were  developed  and  conducted  In  Isolation  from  the  managerial  and 
operational  environments  In  which  Its  results  were  expected  to  be  ultimately  used.  If  a 
project  was  exceptionally  successful  or  caoglit  muiagement*s  fancy.  It  might  find  Its  wiof 
Into  the  working  world;  but.  too  often  the  results  were  too  modest  to  risk  operational 
experimentation  or  were  expressed  In  forms  not  usable  to  managers  and  operations. 

CHANGING  SOURCE  OF  RESEARCH  PROJEa  GENERATION 
Traditional  Picture  of  Ivory  Tower 

Research  Organizations  and  Management/Operations  Elements  Moving 
Toward  Symbiotic  Relationships 

Relationship  Reeded  to  Promote  Long-Run  Efficiency  and  Short-Run 
Survival 


While  this  condition  prevails  to  sexee  extent  today,  the  relationship  appears  to  be 
changing  substantially. 

The  relationship  betweeti  research  organizations  and  their  counterparts  In  operations 
and  management  Is  moving  toward  an  Integral  cooperative,  even  symbiotic,  arrangement. 

While  many  worthwhile  research  projects  and  questions  still  originate  In  the  research  units 
themselves,  jointly  developed  projects  and  management- scopes  projects  are  on  the  Increase. 
Collaboration  In  development  and  testing  of  research  concepts.  Including  procurement.  Is 
useful  not  only  to  promote  efficiency  In  the  long  run,  but  Is  essential  to  sustain  a 
research  capability  In  times  of  tight  budgets  - in  the  short  run. 

A number  of  reasons  exist  for  this  Increased  Interest  by  non- researchers  In  research 
goals,  processes,  and  the  projects  themselves. 
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- . * •>  « REASONS  BEHIND  NON-RESEARCHER  INTEREST  IN 

RESEARCH  GOALS.  PROCESSES,  AND  PRODUCTS 

$ the  reseereh  iii»est*nt  aey  be  subsUntiel  or  the  cost/risk  of 
lepleaenUtlon  ley  be  large 


V.  ; 


The  •glaanor*  of  having  a "research  based"  organization 

Hanagerial  exposure  to  research  tools  and  nethods  in  courses  arid 
■eetings 

Observed  soccess/payoff  of  ca^leted  research  projects 

‘ ' lncr«seda»a«nessby«nag«nt  of  util^^  . 

survival  potential  of  appropriately  applied  research 

• ‘ • . ‘ ^ '}‘  ( 

^ * Variations  in  the  structure  of  research  efforts  within  organizations  tend  to  prw^ 

research  wanagenent  questions  faced  today.  It  is.  in  fact,  the  underlying 
year's  sysposiias. 

OWORTUNITIES  FOR  STIft)LATlN6  USE  OF  PROOIREMEW  RESEARCH 

itostiv  Hindane  but  bear  identifying  and  reexamining.  Traditional  met^s  include 
publiSriJnT^rS:  ^efings.  wsla  and  occasional  "terenc^n  classy 

ATmentioned  earlier  by  General  Sharp.  ^ to  iSeSnt^’ab 

operators  on  full-time  resecrch  teams  and  assign  full-time  researchers  to  lepiement 

tested  Innovations.  I 

teadi^rSilld'l'prS^re^nt  and 

Lr;»  r3.'«s^;2'.rrarf;^  ”*  ““ 

wponuimES  fw  stiiuatik  use 
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Traditional:  Publications,  Briefings,  Symposia,  Courses 

Use  of  Researchers  in  Management 

Use  of  Ibnagers/Operators  in  Research 

Place  Research  Managers  on  Management  Boards 

Develop  Internal /External  Research  Ccwinications  Network 


( ) 
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^ “**  • research  cownicatlon  netMork.  for  convenience  1*11 

bmk  this  letMorfc  Into  two  subsets,  the  Internal  research  coaunicatlons  envlroiaent 
And  tfi6  extAffiAl, 


IWTERJIAl  comincATiows 


. > “ ‘w’  ^ >:/r 

- ' ^ J ;g|v, 


^•"“•jcalljr,  Intenial  research  coannicatlons  present  oore  problcns  than  external. 
C^tltlon  ams  between  departwnts  prevars.  He  still  have  distrust 

a^Jealousy^tecepM  divisions  of  tasks  and  responsiblllt.es  llwits  Infoimtlon  flow 
within  dyartuents.  The  longer  ranges  objectives  of  the  research  elcaent  versus  the 
relatively  shorter  ones  of  other  elemits  are  often  In  conflict, 
i 

•Pprweh  to  Internal  comnlcatlon  Is  to  naoe  one  person  each  frow 

opMtIons  to  each  najor  research  project  whose  Job  It  Is  to 
t**  organization  about  the  projects  alv  and  schedule. 

« ^Itlon  te  providing  guidance  and  logistical  support  to  the  project.  He  try  to  do  this 
^ •“vlsory  group  co^osed  of  such  a wlx.  In  one  case 

!IL!“  project  early,  and  In  another,  we  learned  two  Inportant 

objectives  that  would  have  e^xaped. 

A newsletter  or  periodical  of  organization  - wide  hPPeal  is  a better  place  to  keep 
■enagMeM  and  aerations  Inforwed  of  research  events.  Research  has  shown  greater 
acceptability  of  nore  widely  based  publications  and  greater  readability. 

Cstabllsawent  of  project  officers  or  contect  points  In  field  organizations  who  act  as 
• *||rrogate  researehey  as  yll  as  ewissarles  and  coonunlcators  have  proven  to  be  an  effective 
Channel  of  rapid  Infomtlon  processing  which  can  also  provide  feedback.  In  each  of  our 
owo^ty  coyands.  we  have  alternate  project  officers,  by  nane.  who  11i»  up  data,  gain 

•***;**■  objectives,  and  provide  critical  feedback.  Sow  of  than  ere  here  at  this 
la  the  early  stageSp  this  Is  one  of  the  aost  laportant  resources  available. 

1 coanend  it  to  Oan  Strayer  and  Bob  Judson. 

Inclusimof  a research  session  or  presentation  on  the  agenda  of  the  wajor  staff  or 
director  weting  Is  needed  to  deaonstrate  top  nanageoent  support  and  keep  abreast  of 
research  idilch  way  need  barriers  broken  down  to  be  effectively  ligrlewnted. 

Interal  *terder  boards.*  where  outsiders  are  brought  In  to  observe  the  objectivity 
and  participle  are  extreaely  useful  If  the  organization  Is  open  enough  to  pemit  It. 
not  only  will  this  type  of  confrontation  catch  errors  or  anticipate  probleas.  bwt  will  In 
a dynamic  way  stliulate  other  researchers  and  aanagers  attention  to  related  concepts  In 
their  own  projects. 

IKTERNAL  RESEARCH  COfWUNICATIONS 

Hore  problems  than  external:  Coepetltlon.  accepted  division  of 
responsibility;  long  run  vs.  short  run  objectives 

Name  one  or  more  persons  from  research,  policy,  and  field  operations 
for  each  major  project 

Orgaalzatlon-wlde  newsletter,  periodical 

Field  contact  points  • designated/dedicated 

A 

Inclusion  of  research  activity  on  agenda  of  board  of  directors  meeting 
Internal  ‘murder  boards*  with  outsiders 
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EXTOMAL  OmiNlCATlOlB 
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f OTERML  RESEARCH  amnilCATIOhS 

.Hare  Institutionalized  - Cost  but  Value  Rot  Oaorr-trated  * 

^ Inforwtion  Exchanges.  OooavnUtlon  Centers,  Rosca.ch  Guloes. 

^^*'***^  R®**letters,  and  other  Publications.  Symposia 

' Jo^Jin^eatlon  Media  Necessary.  But  Costly  - Sum  i 

l^y  Be  More  Profitable  Spent  on  Internal  Cownlutlon 

*Scann1^*  Itet  Critical  Aspect  of  External  Conminlcatlon  - " 

Researchers  Tend  to  Ignore  Developments  After  Initial  ChecJis 

REIATIONSHIP  OF  PURSUING  OPPORTUNITIES  TO  P|j^gT_g^fss 

SpleSStltSn^iS  ™iatl2lj  SessfIlTb*lJaJ12rts*tr* 

cite  a fl^wS  we  dlHnd  * category,  but  trill  Just 

ha^  WoHa  thi  ♦L-fi.l?  ” cases,  we  were  not  able  to  do  what  rriaht 

have  aided  the  translation  process  due  to  resource  Haltatlcns  or  lack  of  support.  * 
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Life  Cycle  Costing 
Contractor  Perfomnee  Evaluation  (CPE) 
Desig*  to  Unit  Production  Cost  • 
Harrantles  Usage 
Decision  Criteria  for  CITA  riib 
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The  llnltatlons  on  -sing  procurement  research  are  far  more  nwrwe  than  ^ opportuni- 
ties. Th^  Include  *' . »1c  and  extrinsic  factors.  le  Intrinsic  are  associate  ul  .h 

the  research  Itself  - gualltyp  appropriateness*  t1iael1**2ss,  practicality*  cost  of  inpiewen- 
Utlon.  The  extrinsic  consist  of  legal  and  policy  constraints,  tech«logy  base  bakers, 
organizational  meUVillsa  - or  lack  of  It,  and  failures  of  aanagerlal  nerve  to  nentloi 
Just  a few.  We,  the  researchers  and  managers  can  do  much  to  overco«  the  l^lnslc 
limitations,  for  they  normally  Involve  priorities  of  application  In  the  ihort  run.  out 
the  extrinsic  are  basically  long  run  problems  and  must  await  solutions  developed  on  a 
macro  scale  over  a conslcerable  rerlod  of  time.  11y  experience  has  been  that  talk  too 
«ich  about  the  barrier:  and  limitations  when  It  comes  to  Implementing  procure«nt  re^aron. 
A little  more  time  spert  on  steadily  plying  the  opportunities  may  make  some  of  the  llmiu- 
tlons  disappear*  Before  leaving  this  subject  we  should  all  reinert>er  one  Wng  - h»  Jo 
tell  the  difference  bet.«3n  research  that  cannot  be  Implemented  because  of  llmlU- 

tlons  and  research  which  should  not  be  because  of  Instrinsic  limitations.  If  the  rwearci 
cannot  be  loplenented  for  extrinsic  reasons,  be^nd  the  co?»trol  of 
managomnt,  the  research  Itself  should  not  be  falsely  labelled.  It  may  In  fact  represent 
the  Identification  of  a basic  root  problem.  While  this  should  normally  have  -een  antici- 
pated In  the  research  plannlngTT^tter  later  than  never.  If,  on  the  other  hai^,  the  imple- 
Kntablllty  of  the  research  Is  limited  by  its  quality  or  our  ma  agement  of  the  results, 
we  must  recognize  this  for  what  It  is  and  take  appropriate  action. 
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LIMnATIONS 


IKTRIHSIC 

QiMlltor 

Appropriateness 

Tfaellness 

Practicality 

Cost  of  lopleaerttatlon 


SlfMARY 


EXTRINSIC 

V 

Legal  Barriers 
Policy  Constraints 
O^nlzatlonal  Netabollsa 
Failure  of  Managerial  Nerve 


In  this  syupvsl Mi  I hope  we  will  stress  the  pursuit  of  opportunities  as  our  objective 
and  recognize  llaltatlons  as  temporary  constraints  which  we  will  seek  to  alleviate.  I don't 
mMn  stress  all  success  stories,  but  to  look  at  failures  as  lost  opportunities  which  can 
"*  regairred.  In  closing,  I*d  like  to  pose  several  questions  which  we  can  discuss  at  this 
session  and  throughout  the  sjfHposlun. 
throughout  the  year. 


They  nay  provide  stloulus  for  continuing  enquiries 


Uhat  Methods  of  stii 
are  we  nissing? 


ilating  Increased  used  of  procurenent  research 


Is  the  practice  of  seul-annual  or  quarterly  sittings  of  boards  of 
directors  to  procurenent  research  organizations  effective? 

Can  research  nanagers  and  researchers  nake  meaningful  contributions 
to  top  level  policy  group  meetings?  Uhat  are  the  praictical  drawbacks? 
Uhat  can  we  do  about  then?  ^ ^ 

Does  the  traditional  process  of  publishing  research  results  In 
scholarly  form  actually  Inhibit  their  Inplementatlon?  If  so,  what 
alternative  means  could  be  used  to  gain  usage  and  still  distribute 
the  methodology  and  results  to  others? 

How  can  the  priority  of  research  projects  be  determined  In  situations 
where  procurement  managers  and  researchers  collaborate  on  the  selection 
of  projects? 
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INTROOCXrTION  .. 

Extensive  research  has  been  accoo^lished  in  the  name  of  procurement 
research  over  the  past  few  years  but  no  definitive,  delimited  concept  has 
evolved  as  to  %#hat  constitutes  procurement  research  (12).  A review  of  the 
early  Department  of  Defense  (DOD)  Procurement  Symposia  "Proceedings'* 
indicated  that  professionals  in  the  field  of  procurement  called  for  a 
definition  of  the  term  "Procurement  Research*  as  well  as  the  classifica- 
tion of  its  characteristics  into  a model  to  provide  more  efficient  use  of 
resources*  Therefore,  the  need  exists  to  clearly  define  procurement 
research  and  to  classify  its  characteristics  into  a usable  conceptual  model. 

The  Federal  Government  spends  billions  of  dollars  each  year  for  tha 
procurement  of  the  equipment,  services,  supplies,  and  personnel  required 
for  the  operation  of  the  DOD.  As  a consequence  of  the  magnitude  of  these 
outlays,  repeated  cost,  schedule,  and  performance  problems,  and  increased 
general  public  awareness,  the  DOD  procurement  (acquisition)  process  has  • 
come  under  serious  criticism  in  recent  years  (3:2-4). 

Research  has  been  vie%#ed  by  many  in  the  field  as  a key  to  alleviating 
both  existing  amd  future  procurement  problems  (2:1).  Senator  Stennis, 
Chairman  of  the  Senate  Committee  on  Armed  Services,  and  Congressman  Price, 
Chairman  of  the  House  Committee  on  Armed  Services,  reiterated  this  widely 
held  belief  in  a joint  letter  to  former  Defense  Secretary  Schlesinger: 

We  recognize  the  value  and  importance  of  procurement 
research  as  a means  of  improving  the  procurement  process  — 
one  of  the  most  crucial  tasks  in  Government  (8)  • 

Even  though  there  is  a general  consensus  as  to  the  importance  and 
possible  impact  of  research  on  the  procurement  process,  procurement  research 
as  a discipline  has  not  been  clearly  defined  in  existing  literature  and 
practice.  Also,  there  is  little  agreement  aaong  the  agencies  performing 
this  research  as  to  what  constitutes  research  (1:2).  There  have  been 
several  indications  as  to  a possible  scope  for  procurement  reseairch;  for 
example,  J.  N.  Malloy,  then  Deputy  Assistant  Secretary  of  Defense  for 
Procurement,  described  procurement  research  as  essentially  "a  systematic 
approach"  that  follows  the  scientific  method  (7:215).  However,  there  has 
not  been  a concerted  effort  to  adequately  describe  what  should  be  included. 
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Studies,  Caanission  on  Government  Procurement  (COGP) , stated  that  procure- 


The  result  has  b^en  that  research  e** 
and  diffuse  (8:4).  Robert  Judson. 


■ent  research's 


...  first  order  of  priority  ...  is  to  construct  ... 
* ®®  that  we  can  share  a consensus  on  procurement 

***^^*®®'  ...  a onprehensive  studious  critical  conceptual 
■odel  for  the  acquisition  process  that  will  give  us  insights 

m do  not  now  possess  that  will  help  us  identify  what  we  don't 
know  (5:93). 


The  i^rtai*ce  of  procurement  research  and  the  necessity  of  defining  its 
role  in  government  acquisition  was  reaffirmed  in  an  interview  with 

Administrator,  Office  of  Federal  Procurement 
Policy  (GPPP)  for  (Sontract  Administration.  He  discussed  procument 
research  as  follows: 

I ve  long  had  an  interest  in  procurenent  research.  I 
think  that  it  (an  attempt  to  define  and  classify  procurement 
research)  is  one  that  is  particularly  Important  because  I 
have  Men  a considerable  amount  of  confusion  regarding  what 
constitutes  procurement  research.  I beUeve  that  this  natter 
needs  to  be  clarified  so  that  we  can  more  efficiently  utilize 
manpower  resources  that  we  have  in  this  particular  area 
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years  of  inefficiency  and  redundancy,  the  national  procurement  policy* 
education,  and  research  are  becoming  centralized  and  coordinated  under 
the  FPI. 

'Mr.  Robert  Judson,  then  Deputy  Director,  Coamission  Studies,  OOGP, 
in  an  address  to  the  second  DOD  Procurement  Syn^sium  in  1973  made  a 
challenge  to  the  procurement  profession.  He  said: 

» V 

Tou«  gentlenen,  have  a golden  ofiportunity  to  redirect 
proctirevnt  research  to  achieve  new  goals  of  excellence.  First 
i®t*s  do  oar  research  on  the  of  research  before  we 

lose  the  chance  to  Make  procureaent  trtiat  we  want  it  to  be 
(5:99). 

: ^ i..,r  :i'.:  '>*;•,  » 

This  challenge  was  re-enphasixed  by  Dr.  John  J.  Bennett,  then  Acting 
Assistant  Secretary  of  Defense  (Installations  and  logistics)  in  the 
Defense  Manageaent  Journal.  July  1975: 

Procureaent  research  is  not  yet  a household  phrase  in 
the  Departaent  of  Defense  ....  it  needs  a great  deal  of 
attention  froa  aanageaent  and  those  people  actually  engaged 
in  procureaent  projects  (2:1).  - . . .. 

' ■ ’ 1 ' «i3  . 1 -1 . * * ' 1 % 

In  siBomary,  the  ii^Knrtant  events  in  the  evolution  of  procurement 
research  start  with  the  Second  Hoover  Gcnndssion  in  1953  and  continue  up 
to  the  present  time.  Key  events  in  procurement  reorganization  in  the 
1960*8  include  the  reorganization  of  Air  Force  Systems  Command  and  Air 
Force  Logistics  Oosmand,  the  Hershey  Procurement  Pricing  Conference,  the 
establishment  of  the  Amy  Procuresient  Research  Office,  and  the  Coamission 
on  Government  Procurement.  The  significant  events  (thus  far)  during  the 
1970* s include  the  five  DOD  Procurement  Symposia,  the  establishment  of  the 
Air  Force  Business  Research  Management  Center,  the  addition  of  Graduate 
Procurement  curricula  to  the  Air  Force  Institute  of  Technology,  S^iool 
of  Systems  and  Logistics,  and  to  the  Nav2Ll  Postgraduate  School,  the 
establishment  of  the  Office  of  Federal  Procurement  Policy,  md  the 
founding  of  the  Federal  Procurement  Institute. 

OBJECTIVES  ' - - 


The  following  research  objectives  are  being  pursued  in  this  study: 

1.  To  define  procurement  research  so  that  a common  foundation  can  be 
used  when  discussing  this  subject. 

# 

2.  To  classify  procurement  research  efforts  and  functions  into  various 
areas  and  to  identify  those  areas  that  are  most  frequently  investigated. 

3.  From  these  classifications,  to  suggest  a detailed  £dgoriUsi  whicdi 
can  be  used  for  deciding  whether  an  effort  is  proctirement  research  or  not. 
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METHODOLOGY 


A literature  review  was  initiated.  It  disclosed  an  increasing  interest 
in  the  area  of  procureoient  research  and  in  defining  procurestent  research, 
but  no  suggestions  have  been  made  as  to  how  this  specific  area  of  resear^ 
8.hould  be  defined  or  how  it  should  be  classified  from  a taxonomical  stand- 
point. Content  analysis  provided  a subjective  technique  for  grouping  various 
procurement  efforts:  by  division  of  scientific  study,  by  breadth  of  appli- 
cation, by  degree  of  control,  by  level  of  outcome,  by  level  of  effort,* 
and  by  placement  in  the  acquisition  and  procurement  processes.  Through 
a system  of  suanarizing  and  categorizing,  these  various  groups  can  be 
to  suggest  a definition  for  procurement  research. 

The  research  design  was  divided  into  five  areas: 


1."  Classifying  procurement  research  efforts  and  functions  into 
categories  and  sub-categories.  y 


2e  Ident^lfying  t:he  areas  of  procurement  research  that  were  flK>st 
frequently  investigated. 


i ti  ^ :7 


3.  Defining  procurement  research  in  terms  of  characteristics  idiich 
were  evidenced  in  the  study. 


4.  Suggesting  a taxonony  of  procurement  research. 


/i: 


5.  Designing  an  algorithm  to  use  in  deciding  whether  an  effort  is 
procurement  researcdi  or  not. 


The  first  three  areas  of  the  research  design  were  planned  to  answer 
the  first  and  second  research  objectives.  The  fourth  design  area  was  < 

planned  to  answer  the  second  and  third  objectives  and  the  fifth  design 
area  was  planned  to  answer  the  third  objective.  u.*  ^ ’ 

The  first  design  area  was  planned  to  identify  specific  scientific  and 
research  characteristics  of  procurement  research  as  evidenced  in  the 
"Proceedings."  Through  content  analysis,  the  articles  of  the  symposia 
"Proceedings"  have  been  classified  into  various  categories  and  sub^categories 
of  characteristics.  These  scientific  and  research  characteristics  have  been 
correlated  with  areas  of  the  procurement  and  acquisition  processes. 


To  satisfy  the  first  and  second  objectives,  the  methodological  approach 
of  semantic  content  anadysis  was  adopted.  Prom  the  universe  of  research, 
the  population  called  procurement  research  was  chosen.  This  population  was 
further  narrowed  to  the  sub-population  of  procurement  research  as  reported 
in  the  "Proceedings"  of  the  five  DOD  Procurement  Research  Synqposia.  The 
analysis  consisted  of  a census  of  the  total  sub-population  of  articles  in 
these  "Proceedings." 


*These  categories  were  adopted  from  the  Helmstadter  taxonomy  the 
Strayer  - Lockwood  taxonomy.  See  references  4 and  11  for  further 
discussion. 


u 

u 

u 
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To  validate  the  coding,  a pilot  stiidy  was  accomplished.  TO  enhance 
the  reliability  of  the  research  effort,  a ''t^u^get**  reliability  percentage 
of  90%  was  achieved  during  the  pilot  study.  Additionally,  during  the 
analysis,  random  sasqples  of  articles  coded  by  one  researcher  were  recoded 
a second  researcher  to  insure  consistent  and  standiurd  results,  x . .u 

^ -v  - T 

After  coding  the  data  for  each  "Proceedings,"  a relative  frequency 
count  of  occurrences  under  each  digit  code  was  tabulated.  Each  digit  in  r 
the  seven-digit  code  represented  a category  of  science,  research,  the 
acquisition  process,  or  the  procurement  process.  The  first  digit  was  coded 
to  show  the  division  of  science  used  in  the  research.  The  second  digit  was 
coded  to  show  the  breadth  of  application  of  the  research  te^iniques  used. 
The  third  digit  was  coded  to  identify  the  amount  of  control  used  by  the 
researcher  and  where  the  research  %#as  conducted.  The  fourth  digit  was 
coded  to  determine  the  level  of  outcome  of  the  researcdi  effort;  what  could 
be  said  about  the  area  studied,  did  it  describe  a situation,  or  could  a 
model  be  developed  to  predict  future  events?  The  fifth  digit  was  coded 
to  indicate  the  level  of  effort  used  in  the  research;  i.e.,  the  amount  of 
time  and  depth  of  effort  necessary  to  accomplish  the  research.  The  sixth 
and  seventh  digits  %#ere  coded  to  indicate  the  phases  of  the  acquisition 
and  procurement  processes  with  which  the  research  was  concerned. 

In  the  second  design  area,  the  results  of  the  content  analysis  %«ere 
combined  into  relative  frequency  distributions.  Bach  sub-category  was 
, analyzed  to  determine  those  areas  of  procurement  research  which  %fere  most 
frequently  investigated  and  which  characteristics  %rere  most  prevalent  in 
the  population. 

The  third  research  design  area,  defining  procurement  research  in  terms 
of  characteristics  evidenced  in  the  study  is  being  addressed  at  the  current 
time.  The  characteristics  of  research  and  science  derived  from  content 
analysis  were  combined  with  information  obtained  from  literature  reviews 
• and  personal  interviews  to  develop  a tentative  conceptual  definition  of 
procurement  research.  . 

The  fourth  area  of  research  design  was  planned  to  classify  procurement 
research  efforts  into  various  areas,  as  stated  in  the  second  Objective,  and 
to  suggest  a det£d.led  procurement  taxonomy.  Information  for  this  area  was 
gathered  from  existing  literatxire  and  personal  interviews  (see  Appendix  1). 


The  last  area  of  research  design  W2is  planned  to  determine  whether  an 
effort  is  procurement  research  by  comparison  to  a designed  algorithm.  The 
algorithm  would  be  derived  from  the  taxonomy  suggested  from  the  fourth 
area  of  research  design. 
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«>nf  ”1°“^**  rcMarch  effort  is  not  yet  complete,  initial  data  from- the 
^ have  been  tabulated,  and  an  initial 

taxono^  has  been  developed.  A tentative  algorithm  is  currently  being 

? !**  content  analysis,  which  are  listed  in  Table 

^ showed  the  following  primary  areas  of  eig>hasis  as  related  to  the  selected 
criteria.  ..  ..  ^ ;•  -i.  ^ - T.i- 

r ’ •i  v.jem  ,9c.«nxy4  J.vwer  jijli 

4a*  jiuiix  w»:  *r'- 

f-Pi*  er.tABLE  1 - StMIARy  CT  FINDIBGS  ;i'v.,2  -i  J V-w’i  w 

• .vc'v  .av  ig'J.  .■  i:  ‘ctVTi  r.frr 

1.  Oivision  Of  Science < Social  (45.05%)  .w  v-  • 

JV  --••.»  ' • • - idl-.;,. 

' -irt  ^.-.oL-Ss..!*  & 'UV* Abstract/Social  combined  (45.75%)  ' V -.f  ^'■ 

.ir-jf  u---  . ,,  a....'.!.-.-*;  j-A 

2.  Breadth  of  Applicationt  Applied  (61.26)  > t • il,  its  ,347  2toi‘W 

",11.  •■  •’•'•-■  ‘ >•  V 

3.  ’ Degree  of  Control:  Library  (52.25%)  - '-i-?  • a.'/rv.j  . :j,vv  j*5v.b;sf. 


^a.*rr?5it<;^Tn  *-jk» 


^-•.1 


- ‘ ' ''  . -•  r.  f • . 

4.  Level  of  (Aitcome:  Descriptive  (55.66%) 

► . . .-c^/v.  

5. 1 level  of  Effort:  Professional  Pjq>er/Besearch  Monograph  (68.46%) 

6.  Phase  of  the  Acqnisitioo  Process:  •.•  . . ..\r;..  r .■ 

• • ‘ - Z ^ * 

More  Than  One  Phase  (37.83%)  ‘ / 

; r.  U< 

Not  OcsDceraed  with  the  Acquisition  Process  (46.85%) 

’ " ‘ * - « 
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7.  Phase  of  the  Procurement  Process 

Pre>award  (45.95%) 

More  Than  One  Phase  (33.33%) 


■ 1*'  ''  I 

i K-  ^ 


OI  N.. 


t-w 


‘•X^.  v-r: 


■ As  corollary  information  to  the  content  analysis,  the  researdiers 
n^ed  the  source  of  each  article.  Cf  the  one  hundred  eleven  (111)  articles, 
the  source  distribution  is  recorded  in  Table  2. 
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TABLE  2 - ARTICLE  SOURCE  DISTRIBUTION 


. I 


AGENCY 

NUMBER 

PERCENTAGE 

DOD 

„ w.‘  • 

8 

7.21% 

AIR  FORCE  * ' 

* ^ S * 'J*.  - . .1  "* 

38 

t- 

^ 34.23%  - 

*.  rw  ^ 

ARMY  * 

19 

? } k . 

•’*  • 17.12%  • --f 

NAVY  ‘ 

14 

•r:  rrr  - 12.61%  ^ 

NOH-DOD  ’ 

i ‘ j.  i . * ' . 

..  i.-.  " 

> ^ -.'.'i  7 ./i-T  I' *rr ; ; t r ^ ^ n 

Govenment 

■ ' 11 

9.91%  - 

Private  Business/ 

" Universities 

21 

. n.  ;•  .r.K 

18.92%  • 

V . 


The  initial  taxoncsy  vas  divided  into  five  levels  of  procnreaent 
re8e2u:ch.  Bach  level  subdivided  the  previous  level  into  nore  specific 
areas  where  procurement  research  can  be  identified. 


TENTATIVE  CONCLUSIONS 


Current  progress  in  the  research  effort  has  suggested  tentative 
answers  to  the  research  objectives.  The  results  of  the  research  to  date 
suggest  the  following  definition  of  procurement  research: 

Procurement  Research  (and  Acquisition  Research)  is  an 
applied  science  using  the  characteristics  of  the  social  sciences 
in  oonbinatiou  with  abstract  or  mathematical  sciences  to  solve 
procurement  problems.  It  tends  to  rely  heavily  on  the  use  of 
previously  gathered  data  (library  research)  to  see)c  solutions 
to  problems,  equally  dividing  its  efforts  between  the  acqui- 
sition and  procurement  processes,  with  major  emphasis  in  the 
procurement  process  in  the  pre-a%#ard  phase  in  an  effort  to 
identify  cost-related  problems  in  the  procurement  of  either 
major  weapon  systems  or  non-weapon  systems. 

The  results  of  the  content  analysis  are  limited  to  the  actual  sub- 
population itself,  but  the  sub-population  of  the  "Proceedings"  represents 
an  important  cross-section  of  recent  DOD  procurement  research  experience. 
Information  derived  from  the  analysis  of  this  s\2b-population  can  suggest 
important  characteristics  and  relationships  of  other  procurement  research 
efforts. 
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•^e  areas  of  emphasis  in  the  procurement  research  of  the  "Proceedings" 
wre  Identified  in  the  content  analysis.  Procurement  research  was  charac- 
terized  ^ a social  science  with  abstract  science  combined  more  often  than 
not.  Efforts  were  primarily  applied  to  solving  problems.  The  research 
was  primarily  done  through  a selected  aggregation  of  information  (library) 
^ the  level  of  outcome  was  descriptive.  In  the  sub-population,  the 
^el  of  effort  was  primarily  a professional  paper/research  monograph. 

The  relationship  of  procurement  research  with  the  acquisition  process 
sto^  that  efforts  generally  involved  more  than  one  phase  of  the  acqui- 
^ion  process  or  were  not  concerned  with  the  acquisition  process  at  all. 
^hasis  in  tte  procurement  process  was  primarily  in  the  pre-award 
with  many  articles  dealing  with  more  than  one  phase. 

The  definition  of  procurement  researdi  and  the  classification  of  the 
diarac^istics  of  procurement  research  were  combined  with  the  informaUon 
rom  the  interviews  (6,  10,  13)  and  the  literature  review  to  develop  the 
present  taxonomy.  The  emphasis  for  the  taxonomy  has  been  to  cover  all 
possible  areas  of  procurement  research  as  suggested  by  various  informaUon 

TOurces.  A t^tative  taxonomy  »d»ich  has  been  developed  by  the  researchers 
rerlected  in  Appendix  1. 

The  rese^ch  effort  will  be  adjusted  as  the  researchers  discover  new 
^ormatiOT  that  impacts  on  the  definition,  the  taxonomy,  or  the  algorithm. 
Current  plans  include  a more  detailed  study  of  the  results  of  the  content 
^ysis  to  show  possible  shifts  in  areas  of  interest  within  the  acquisi- 
tion ^procurement  processes;  the  development  of  a sixth  level  (where 
possible)  in  the  taxonomy,  such  as  COST  KSIGN  tdiich  is  on  the  fifth  level 
mder  COST/PRICE  ANALYSIS  under  the  SOLICITATION/EVALUATION  Cycle  in  the 
RE-AHARD  PHASE.  The  next  level  (sixth) 'under  COST  DESIGN  would  be  areas 
wch  as  Life  Cycle  Costing,  Design  to  Cost.  Lastly,  refinement  to  improve 
ccuracy  and  reliability  of  all  areas  of  the  research  effort  will  be 
accomplished. 
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Standards  are  eoer^ng  both  fro*  within  agencies  conducting  research  and  froa 
legislative  branch  reviews  of  agencies  activities. 

Miat  has  led  the  legislative  branch  to  review  these  research  activities?  General )y. 
It  has  been  concluded  that  reliance  on  functional  policy  specialists,  ad  hoc  aanageoent 
fixes  and  regulatory  aechanisas  to  Identify  and  solve  chronic  probleas  and  nake  sub> 
stantive  Inprovenents  in  the  procurenent  area  has  not  worked.  It  was  concluded  that  a 
leading  candidate  as  an  alternative  to  these  previous  efforts  Is  an  organized  approach 
to  procurenent  research. 

Again,  why  should  the  legislative  branch  be  concerned  with  the  deUlts  of  procure- 
■ent  research  prograns?  It  was  concluded  that  necessary  pollqf  redirections  require  a 
series  of  operational  changes  In  agency  procurenent  and  acquisition  practices  which 
touch  both  legislative  and  executive  branch  responsibilities. 

Considerations  regarding  standards  for  the  conduct  of  research:  Basic  definitions 

critical  Investigation  Into  an  agency's  procurenent  and  acquisition  pro- 
cesses  with  the  objective  of  leproving  their  effectiveness;  Includes  (^velopoent  of 
new  procedures,  techniques  and  approaches,  as  well  as  experlnentatlon  ained  at  gaining  / 
new  Insights  or  naklng  new  discoveries  about  these  processes.  v 

toulsitlon  usually  used  In  connection  with  major  systens  and  Includes  the  entire 
spectrum  of  activities  starting  with  an  agency's  determination  that  It  needs  a new 
or  i^roved  capability  for  an  assigned  mission  through  to  operational  use.  (This  defi- 
nition Is  consistent  with  new  standards  and  areas  for  the  conduct  of  research  suggested 
by  Oe  Circular  A-109  and  revised  000  Oirectives  5000.1  and  .2.  The  new  milestone  0 
to  1 sequence  Involves  ‘maiuging  the  determination  of  what  we  need.*  The  Integrity  of 
pursuing  this  decision  making  process  should  be  guided  in  part  by  the  business  management 
judgments  from  the  ‘procurement  community.*  This  is  a role  from  which  the  procurement 
coonunity  has  traditionally  excluded  Itself.) 

Procurement  is  either  ‘independent*  or  an  iaportant  subset  of  the  acquisition  pro- 
cess.  As  an  independent  process,  it  starts  when  a procurement  request  Is  received  In  ‘ 
a purchasing  office  and  continues  through  award  and  administration. 

The  distinction  between  acquisition  and  procurement  does  not  degrade  the  Importance 
or  the  other,  txit  it  Is  a useful  distinction  when  examining  whether  we  are  con- 
ducting relevant  research  in  both  areas. 

Standards  for  expectations  of  what  research  ought  to  provide. 

!•  basis  for  problem  identification  and  problem  solving;  these  should  be  the  underlying, 
root  cause  problems  and  research  should  distinguish  cause  and  effect  relationships  in 
problem  identification. 
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Z.  basis  for  a corporate  wemory;  a "lessons-leamed"  clearing-house  is  required  as 
part  of  the  data  base  for  the  conduct  of  research. 

3.  basis  for  developing  procedures  for  new  iwlicies;  before  the  issuance  of  new 
policies,  alternative  a^roaches  should  be  identified,  explored  and  tested. 

4.  indyendent  research,  looking  to  long-range.  Innovative  iq>roveaents , not  con- 
strained  by  Innediate  problem  solving. 

5.  support  for  the  deyelcpnient  of  education  and  trairtm  ■rooraas;  what  organizations 
are  concerned,  or  should  be.  with  standards  for  the  coMim  or  research. 

Office  of  Federal  Procuraant  Policy 

Itole;  Govement-wlde  research  leadership  and  coordination 

As  required  by  P.L.  9J-400,  OFPP  has  a responsibility  for  'proaoting  and  con- 
ducting research  in  procureoent  policies."  It  will  be  acting  in  concert  with  the 
Federal  Procureaent  Institute  and  has  already  initiated  soae  aajor  systems  research 
activities. 

Federal  Procureaent  Institute  .«'  ■ 

Role;  "Proaote.  monitor  and  conduct  research  to  develop  business  methods  and 
management  techniques  that  will  advance  the  state  of  the  art  In  procureaent." 

The  principal  objectfw  should  be  to  encourage  and  support  existing  prograas  but 
FPI  take  research  Initiatives  where  gaps  or  omissions  are  determined  to  exist  in 
existing  research  coverage. 

Acquisition  Executives 

tele:  Should  have  overall  research  responsibility  and  guidance  within  an 
individual  service  (agency). 

The  acquisition  executive,  required  to  be  designated  under  (MB  Circular  A-109, 
would  be  a logical  focal  point  for  agency  research  responsILiilltles  and  guidance. 

Agency  Operating  Elements 

Agency  Research  Activities 

The  traditional  relationship  of  sponsor  and  performance  would  continue. 

Standards  for  basic  research  capabilities 

1.  Establishment  of  a clearing-house  for  the  Identification  and  sharing  of  research 
data.  This  could  be  covered  by  the  FPI  and  OFPP. 

2.  Establishment  of  an  information  data  base,  including:  bibliographies,  abstracts, 
studies  and  reports  together  with  directories  of  Individuals,  associations,  pro- 
fessional societies  and  their  journals,  completed  research  of  both  the  public  and 
private  sectors. 
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3.  Establishment  of  a professional  research  cadre  with  a mix  of  professional 
backgrounds  and  experience. 


4.  Effective  access  to  research  users  and  procurement  practitioners. 

V ji, 

5.  Establishment  of  a system  for  developing  and  refining  long  range  research  areas 

of  concern,  and  allocating  resources  to  these  areas.  . 

‘ 6.  Es^llsh  sustained  support  and  sponsorship  fron  top  aviageaent  and  Middle  level 
.^operating  eieaents  for  a progrw  of  procureKnt  research. 
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WHAT  ARE  WE  BUYING  HERE? 


Daniel  E.  Strayer.  Lt  Col,  USAF,  Ph.D. 

Lyle  U.  LockMood,  NaJ,  USAF 
Air  Force  Business  Research  Management  Center 


"Ihe  time  has  coae,"  the  Walrus  said, 

] "To  talk  of  nany  things: 

. Of  shoes — and  ships— and  sealing  wa*— 

Of  cabbages— and  kings— 

^ hortm  CarxoU 

PART  I - IMTRODUCnOH 

^ Acquiring  complex  systems  Involves  the  need  to  speak  of  many  of  the 
Items  referred  to  in  Lewis  Carroll's  famous  quotation.  Similar  techniques 
I and  approaches  are  enployed  to  buy  a vast  range  of  products.  Indeed,  it 

— is  precisely  this  variety  of  acquisition  concerns  which  has  caused  us  the 
gravest  difficulties  in  the  past,  and  continues  to  plague  and  thwart  us. 

< Although  the  system  acquisition  process  has  long  been  the  topic  of  high 
^ — level  study,  we  do  not  yet  have  anything  approaching  a consensus  as  to 
what  ought  to  be  done.  In  fact  when  the  range  of  concerns  is  boiled  down 
.to  the  essentials,  it  is  no  exaggeration  to  observe  that  requirements  man- 
V agement  is  one  of  the  most  difficult  problems  facing  today's  acquisition 

— managers. 

( For  most  complex  itans  we  do  not  presently  enjoy  a clear  and  generally 

agreed  on  definition  of  requirements.  This  paper  is  dedicated  to  the  pro- 
position that  the  next  substantial  improvement  in  the  management  of  weapon 
acquisition  process  must  come  from  improved  understanding,  and  ultimately 
rigorous  management  of  requirements  themselves.  We  can  no  longer. reljr  on 
gross  control  of  the  total  flow  going  through  the  system.  To  be  effective, 
we  must  manage  the  contents  of  our  requirements  pipelinel 


PART  II  - THE  REQUIREMEHTS  ESTABLISHMENT  PROCESS 


Consider  the  graphical  representation  of  the  USAF  requirements  process 
set  forth  in  Figure  1.  (While  the  USAF  process  is  used  as  an  example  here, 
it  is  considered  reoresentative  of  the  entire  DOD  acquisition  requirements 
generation  process.) 
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Figure  1 


Figure  1 illustrates  the  nanasenent  system  that  emphasizes  primar- 
ily one  portion  of  the  requirement — the  operational  n^d  for  the  acqui- 
sition. The  elaborate  system  of  checks,  balances,  and  consultation 
exists  to  satisfy  that  need.  All  of  the  steps  in  the  process  lead  to 
a common  end— establishnent  of  a program  to  satisfy  the  operational  need. 


There  is  no  arguaent  with  the  essentiality  of  this  type  of  review 
system.  However,  it  is  now  time  to  address  other  components  of  the 
problem.  The  existing  process  turns  on  the  main  valve  (the  requirement 
itself)  but  it  does  not  yet  address,  in  sufficient  detail,  what  shall 
be  included  in  the  content  pipeline.  A clearer  understanding  o^  what 
is  included  in  requirements  must  be  established  if  life  cycle  cost, 
design- to-cost,  and  other  objectives  are  to  be  met.  Only  then  can  we 
clearly  understand  exactly  what  we  are  buying.  And  when  we  do  that, 
we  will  be  in  a position  to  manage  the  introduction  of  requirements 
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into  the  contract  and  thus  to  greatly  improve  overall  performance.  Let 
us  consider  how  requirements  might  be  more  precisely  categorized  so  as 
to  permit  a more  refined  managerial  approach. 

v-  * . rf  ' . . ^ . .'1  'j_j  i jjifi 

• .J.  ."‘f  . - . • ' JH 

■Tii  _ ..  , '.f-  j . edr'szeh 

' PART  III  - A NEW  REQUIREICNTS  CATEGORIZATION  ■ 

lO  V;  ^ . ::cj-  i'ty.'iiO- 

^ ■ •i*' •*  -i**  V : i ^ * * * ' 'U' ^ ^ ‘ V 

The  total  system  requirements  flovfng  through  the  process  pipeline 
can  be  divided  into  six  distinct  categories.  These  categories  are: 

- .vC.  w w 

• • ofsvsb  b»%r; 

“i-  ?, 

-•■  * 'I'".  . 


a.  HI sslon  requirements. 


. b.  Operating  characteristics. 


e.  Legal  obligations. 


Design  standards  and  specifications.  • ' /.  j 
,d.  Management  systems  standards  and  specifications.  -o 

■■■  ‘ 'i:  . V ;C  <<•  1. 

_ - ^ 1 ■"-•'V  r.-  Nict  7?o 

i . f.  Programing  requirements.  • i.;.-  rvis 

All  of  these  are  coanunlcated  to  Industry  by  the  procurement  and  program 
management  process  end  create  both  the  administrative  and  the  technical 
environment  In  which  the  government's  need,  I.e.,  requirements,  win  be 
fulfilled  by  the  contractor.  Although  they  are  considered  contractually 
equal,  they  are  In  fact  quite  different. 

Mission  ryjulrements  ultimately  quantify  the  need  fbr  acquisition. 
Included  in  this  category  are  functional  definitions,  e.g.,  transportation 
of  troops,  cargo;  destruction  of  targets;  transmission  of  messages;  etc. 

Also  Included  In  this  category  are  surrogates  for  functions  conaonly 
called  performance  parameters.  Exan^les  of  these  are:  speed,  range, 
altitude,  capacity,  effectiveness,  accuracy,  etc.  In  total,  the  mission 
requirements  define  the  purpose  of  the  system.  They  spell  out  what  the 
system  is  expected  to  accomplish.  They  deal  with  accoaplishment  In  the 
mission  performance  mode,  that  Is,  In  a brief,  usually  mission-defined, 
tine  span.  Thus  they  are  almost  alwa3rs  measured  Instantaneously  during 
the  test  and  operating  modes.  Measurement,  and  therefore  evaluation, 
can  be  both  rapid  and  reasonably  accurate. 

Operating  characteristic  requirements  quantify  many  of  the  efficiency 
Indicators  of  the  system.  They  ircl ude  much  longer  time  consideration 
because  they  combine  the  functional  components  of  life  cycle  cost- 
reliability,  mainto inability,  quantity  and  quality  of  operators, 
expected  useful  life,  logistics  support,  and  component  interchangeability 
standards.  These  requirements  impact  on  the  system's  design.  However, 
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they  are  not  usually  measurable  at  the  same  time  that  mission  requirements 
are  measured.  The  success  of  satisfying  such  life  cycle  considerations 
Is  measurable  only  over  time,  frequently  a rather  long  time  continuum. 

Design  s^ndards  and  specifications  deal  with  the  transformation  of 
mission  requirements  and  operating  characteristics  Into  hardware.  They 
describe  specific  knowledge  of  measurement  Inputs  Into  the  design  process. 
Included  In  this  category  Is  the  stated  order  of  preference  for  specifi- 
cations and  standards— components,  material,  and  processes.  The  onhr  of 
preference  results  from  the  belief  that  specifications  and  standard  are 
the  technical  corporate  body  of  knowledge.  They  are  codified  lessons 
learned.  As  such,  they  can  become  inflexible  guidelines  or  directives 
to  the  contractor.  He  impose  them  as  design  constraints  in  order  to 
avoid  new  development  costs,  assure  standardization,  strive  for  co^ietl- 
tive  procurement  of  homogeneous  products,  and  avoid  costs  of  nonstan^rd 
components.  All  of  these  are  worti^  and  desirable  goals. 

Hanayment  sys^s  spedfi<ations  and  standards  either  specify  the 
nature  of  an  organizational  behavior  pattern  or  require  the  disclosure  • 
of  specific  managerial  information.  This  category  is  exemplified  by 
such  things  as  program  management  requirements,  system  engineering  man- 
agement plans,  reliability  program  plans,  configuration  management 
plans,  cost  schedule  control  system,  and  the  like.  The  purpose  of 
each  category  in  this  requirement  is  coanon;  to  elicit  a desired  level 
of  contractor  behavioral  or  managerial  response. 

Legal  obllMtions  include  both  mandatory  and  bilateral  requirements 
that  are  placed  on  the  contractor  and  the  government  program  office  by 
basic  contract  law,  federal  law,  or  agency  regulations.  Legal  require- 
ments are  designed  to  accomplish  various  national  and  program  management 
objectives.  These  have  various  political,  economic,  technical,  or  social 
dimensions.  Examples  are  many  and  include  the  Valsh-Healey  Act,  OSHA, 
environmental  protection  regulations,  equal  employment  opportunity 
regulations,  the  cost  accounting  standards,  and  many  more.  In  addition 
to  the  legal  obligations  mandated  by  law,  bilateral  requirements  are 
frequently  agreed  on  by  the  contracting  parties  and  include  ^e  of 
contract,  method  of  payment,  restitution,  warranties,  corrections  of 
deficiencies,  govemnent-fumished  property  or  services,  forward  pricing 
agreenmnts,  adjustments  for  abnormal  price  escalation,  and  the  like. 


Proqraimiinq  reoul laments  are  allocations  of  total  program  costs  and 
quantities  into  annual  or  other  periodic  partitions.  These  are  usually 
described  in  terns  of  funding  ceilings,  time-phased  budgets,  and  delivery 
schedules.  In  an  uiconstrained  mode,  these  requirements  are  a statement 
of  when  the  mission  need  must  be  satisfied.  These  requirements  are 
initially  defined  by  the  using  command  and  modified  by  planning  staffs 
and  development  agencies.  Further  modification  or  adjustment  of  pro- 
gramming requirements  are  made  throughout  the  federal  budgetary  process. 
The  resolution  of  progranining  requirements  and  mission  requirements  has 
been  the  focus  of  annual  debate  at  the  national  level.  Host  of  this 
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audience  Is  well  aware  of  the  attention  given  to  these  requirements  by 
the  full  cooplement  of  acquisition  managers  and  staffs. 

'Figure  2 breaks  down  the  systems  requirements  Into  the  six  categories 
Just  described.  Note  that  these  are  both  primary  objectives  and  second- 
ary  objectives  and  can  thus  be  referred  to  as  needs  on  the  one  hand  and 
constraints  on  the  other.  By  adopting  the  needs  and  constraints  view- 
point, we  can  better  understand  how  the  conflicts  so  frequently  observed 
are  grated  ^ sustained.  , 
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PART  IV  - WHAT  REQUIREHENTS  BUY 


( 


If  requirements  did  nothing  more  than  sit  In  the  contract.  It  aright 
not  be  too  bad.  However,  each  category  of  requirements  generates  actions 
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on  the  part  of  the  contractor,  reactions  on  the  part  of  the  covemment 
both.  Each  requirement  in  each  category  causes  a flow  * 
and  the  contractor.  A requirement  is  coimnunicated 
requirement  into  hardware  or  infor- 

nation  and  returns  that  to  the  govensMnt.  The  government  then  returns 
^^r  money  or  Information  to  the  contractor,  and  so  the  prtxxss  goes 

U 1^®"  fl?°  ”'^“7  functions  with  thS  infonnSion 

eaploys  It  as  an  input  to  operations  or  uses  the 

.III.?*  control  ongoing  operations.  Requirements 

9^nerate  actions • and  actions  cost  noney. 

^fJPjes  of  each  category  of  utilization  abound.  A use  function  is 
ifi^® t^i.****®®  Training  Comnd  initiates  a training  program 

to  provide  the  Ru^tity  and  quality  of  operators  that  the  contractor's 
inforaiation  says  will  be  needed  to  operate  the  system  at  the  level 
required.  Here  the  systems  operating  characteristics,  translated  into 
manpower  requirenents,  become  the  source  of  information  which  keys  the 
^vernmnt  s training  fioictions.  In  other  words,  the  government  uses 
m information  provid^  by  the  contractor  to  perform  additional  actions 
that  are  essential  if  the  system  is  to  be  utilized  as  planned.  Control 
is  exercised  somewhat  differently.  The  contractor  is  requirwl  to  provide, 
i?r  assurance  progran  to  satisfy  the  requiranents  of 

'“'«^®58A.  The  rontractor,  after  designing  and  implementing  a program 
to  sattsfy  the  quality  assurance  requirement,  asks  the  plant  representa- 
ttve  to  reviw  ft.  The  plant  representative  is  responsible  for  assuring 
that  the  contractor  s plan  conpiles  with  the  government's  need  as 
expressed  by  the  standard. 


In  both  cf rcunstanoes,  cost  and  schedule  impacts  are  associated. 
Indeed,  it  can  be  argued  that  cost  and  schedule  can  operate  as  an 
objective,  i.e.,  a requirement,  or  as  a constraint  depending  upon  the 
situation.  However,  each  requironent  category  clearly  consines  both 
resources  and  time  from  both  the  contractor  and  ultimately  the  govern- 
ment. Good  management,  therefore,  should  establish  the  relative  cost 
ve^us  benefit  of  eech  T^equirenent  in  order  to  evdluete  its  net  con- 
mbution.  Yetp  as  presently  organized,  management  cannot  review  the 
total  requirements  and  assess  their  costs  and  benefits  before  they  are 
placed  on  toe  contractor.  In  fact,  there  is  no  entity  which  is  respon- 
sible for  doing  so! 


PART  V - HOW  TO  BUY  REQUIREMENTS 


■ If  this  description  of  requirements  categories  is  true,  and  all 
requirements  can  be  satisfactorily  categorized  accordingly  (and  several 
practitioners  have  assured  us  that  that  is  the  case),  the  next  question 
to  be  addressed  is  that  of  usage.  How  siiould  we  buy  requirements? 
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Clearly  this  is  a time  sensitive  process.  Various  requirements  have 
their  primary  roles  at  different  times  in  the  development  process. 

Hisslon  and  performance  requirements  concentrate  heavily  on  the  front- 
end.  prior  to  OSARC  I,  DSARC  0.  Operating  characteristics  tend  to  be 
mor«  heavily  represented  In  the  operation  and  support  phase  of  the 
Neapcn  system's  existence.  The  contractual  terns  and  conditions  exist 
throughout  the  contractual  relationship  while  the  management  systems 
specifications  and  standards  tend  to  be  more  Important  during  the  full- 
scale  development  and  production  phases.  .. , '•  ^ 

, By  employing  the  requirement  categories  to  provide  a framework  man- 
agement can:  - - 

a.  Identify  the  total  set  of  requirements  that  will  be  applicable 
to  an  acquisition  program— and  more  Importantly  determine  which  will  not 


be  used. 


i 


■'  b.  Determine  the  appropriate  program  phase  for  1ntro<hic1ng  com- 
binations of 'requirements  that  will  be  applicable.  j t 


) 


c.  Assign  responsibility  to  the  approprtate  organisational 


elements  for  evaluating  the  requirements  under  their  control. 

d.  Assign  cost  and  worth  values  to  each  requirement,  and/or  , 

requirement  category*  and  . 

* t 

e.  Analyze,  In  advance,  the  trade-offs  and  resolve  the  cwfllcts 
between  short-range  and  long-range  costs  and  benefits  associated  with 
requirements. 

m principle.  It  may  be  possible  to  so  categorize  requlrronts  as  to 
require  much  less  during  the  conceptual  phase  building  gradually  towards 
full  Imposition  of  the  applicable  requirements  when  the  system  enters 
production.  Such  phase  requirement  management  could  stimulate  Inrova- 
tlon.  In  both  technical  and  management  aspects  of  sys^s  acquisition, 
as  Meil  as  improving  resource  utilization.  Figure  3 Illustrates  how 
this  might  be  conceived.  , . , 
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Figure  3 

Phased  Requi resent  Introduction 
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PART  VI  > OPPORTUilTIES  AND  CHALLENGES 


Althou^  top  Management  has  placed  increasing  emphasis  on  various 
requirements  aspects  over  recent  years,  much  remains  to  be  done.  Speci- 
fication tailoring,  use  of  connercial  equivalents,  and  other  approaches 
have  yet  to  be  fully  developed.  In  short,  there  are  many  opportunities 
for  management  In  the  requirements  dilenina.  A first,  and  necessary  step 
Is  a validation  of  the  scheme  just  outlined.  Management  can  take  the 
Initiative  in  arranging  for  a meaningful  test.  Assuming  that  that  is 
successful,  and  we  believe  ft  would  be,  management  can  then  properly 
assess  the  institutional  approaches  to  its  proper  utilization.  Researchers 
can,  of  course,  assist  at  each  step  of  the  way. 


From  a research  viewpoint,  the  requirements  process  is  largely 
unexplored.  Our  review  of  the  published  literature,  and  our  discussions 
with  knowledgeable  managers,  revealed  less  than  ten  studies  which  deal 
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dtrectly  with  the  requirements  measurement  issue.  None  of  these  are 
substantiated  by  either  carefully  conducted  case  studies  or  quant1ta> 
tive  research.  Thus,  while  the  process  of  generating  the  requirement, 
expressed  in  terns  of  need.  In  the  management  review  of  acquisition 
perfonoance  has  been  studied,  it  is  a fact  that  the  substance  of 
; requirenents— their  form,  costs,  and  benefits,  etc.»have  yet  to  be 
subjected  to  serious  Inquiry.  • ..v.  , ...  v,. 

( Serious  inquiry  and  ultinately  rigorous  management  of  system  require*  ' . «' 
' ments  can  help  us  insure  that  we  get  what  we  need— and  no  more.  And  that  * 

would  help  us  avoid  figurative  experience  with  the  remainder  of 
Lewis  Carroll's  quotation  which  b^an  this  paper: 


. ' **And  whj  the  sea  la  boiling  hot 
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the  use  of  formal  govenwnt  specifications  In  advertised  procurenents • 


I^wrtent  minies  often  overtake  the  othervfse  desirable  objectives  to  be  obtained  by 

At  TArasAl  #4^  4^  — a a ^ ^ ^ 


Another  reality  has  been  that  Initial  bid  price  can  be  a Misleading  Indicator  of  ultlnte 


90ve«"ent.  Wille  existing  procedures  routinely  anticipate  aMirds  based  on 
cons^ratlons  other  than  bid  price  alone,  the  preponderance  of  advertised  anards  are  "He 


on  the  basis  of  Iom  bid. 


Proposals  have  been  wde  to  stllia  functional  purchase, descriptions  as  an  alternative 
to  costly  and  restrictive  fornel  gover—nt  specifications.^  Such  proposals  cany  the 

neetlng  user  needs  at  a least  uUlnitc  cost.  These  sane  proposals  also 
raise  difficult  auerd  deternination  considerations. 

Research  suggests  that:  ^ ‘ 

1.  Guidelines  exist  for  establishing  valid  functional  purchase  descriptions. 

• * 

2.  Functional  purchase  descriptions -can  be  reconciled  with  equitable  award  deternina- 
tion proceihires  based  on  a "least  cost"  standard  unlch  can  include  both  bid  and  user  coste. 

3.  Functional  purchase  descriptions  have  the  potential  to  lower  user  costs  for 
particular  kinds  of  equipnent  purchases,  as  well  as  lower  or  ellninatc  the  costs  associated 
with  writing  and  aikalnlsterlng  fornal  governnent  specifications. 

4.  Conpetitlon  can  be  Increased  and  nade  a wore  useful  device  by  using  functional 
purchase  descriptions  to  elicit  total  product  conpetitlon  rather  than  the  narrow  Initial  price 
cowpotitlon  associated  with  the  use  of  fornal  governnent  specifications. 


Sg€  PROBLEMS 


User  Needs  and  Costs 

Procurront  systeas  which  lifiose  nandatory  sources  of  supply,  fornal  governnent  speclfl- 
utlons.  directed  procurenent  nethods  and  other  restrictive  procedures  may  nake  it  difficult 
for  a user  to  satisfy  actual  needs. 


H««ver.  even  under  todays  procedures,  where  extensive  documentation  Is  required  for  all 
exceptions  to  formal  advertising,  a user  often  can  obtain  just  what  Is  needed  If  there  is 
enough  tine  and  persistence  to  prevail  in  the  face  of  procurenent  complexities. 
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For  advertised  procureKnts.  it  1$  wll  esUbllshed  that  Federal  agencies  can: 
detendne  their  needs  ^ 

-use  "restrictive”  specifications  . r ^ 

• consider  factors  other  than  price  In  detendnlng  award^  • 

The  precedence  of  GAO  decisions  granting  this  latitude  suggests  a flexablllty  In  dKlsIon 
Mking  ^Ich  Is^not  characteristic  of  actual  practice.  The  hassle  of  reviews,  documentation, 
tine  telays,  and  limited  mffbers  of  procurement  personnel  do  not  make  actual  need  concerns  a 
readily  acceptable  decision  standard  for  the  average  procurement  situation. 

In  evaluating  the  present  procurement  systoa.  the  cost  and  complexity  of  communicating 
nee«  wt  be  considered.  A sl^ilflcant  part  of  this  cost  tfpically  resides  In  the  develop- 
ment of  formal  government  specifications,  which  almost  always  trail  the  developrent  of 
commercial  product  counterparts.^ 

— ’ • . - 

< *<5rove*ents  or  sioplicatinis*  added  to  user  requests  by  review  authorities  result 

in  prtHteU  which  will  not  weet  actual  user  needs.  Fbiwal  specifications  and  fomal  state- 
■ents  of  requireaents  tend  to  becoee  cluttered  with  clauses  that  often  do  not  even  provide 
an  adequate  basis  fbr  bidding.* 

• ^ r 

Another  ieportant  aspect  of  evaluating  our  present  procureoent  syv.tcai  is  to  note  that  the 
toUl  c«t  of  satisfying  user  requiroMts  is  directly  affected  by  the  tiweliness  of  delivery 
M product,  (hir  econoqy,  which  now  places  a preeriuM  on  labor,  weans  that  lo^s> 

tics  concerns  wist  consider  the  cost  of  idle  personnel  and  equipwent  caused  by  late  delivery. 
It  also  weans  that  life  cycle  costs  involved  with  naintcnance,  service,  downtiwe,  and  repair 
account  for  the  nejoi^ty  of  all  costs  associated  with  procuionents. 

In  the  face  of  these  cost  realities.  Hwiting  buyer  choice  to  initial  bid  pHces  is  naive 
and  unnecessary.  The  nanagewent  of  sealed  bid  procurewents  oust  nature  to  consider  the  cost 
realities  overtaking  advertised  procurewents. 

Specifications^ 

There  are  approxiutely  40,000  federal  specifications  and  industry  standards  used  for 
federal  procurewents.®  Itany  of  these  specifications  are  obsolete. 

Virtually  all  specifications  cite  other  specifications  and  publications  and  Incorporate 
then  by  reference.  For  exawple.  the  Cownission  on  Govervwnt  Procurewent  identified  113 
specifications  for  the  procurewent  of  an  ordinary  light  bulb.^ 

Other  specification-related  problews  cited  by  the  COGP  included: 

- Purchase  under  a forwal  specification  where  cooKrclal  products  are  available 
usually  results  in  greater  cost  to  the  govemwent. 

- Use  of  forwal  specifications  way  deny  the  govemwent  current  technology. 

” Strict  acteinistration  of  specifications  for  cowKrcial  products  discourages  potential 
suppliers,  thus  restricting  competition. 

“ Specifications  establish  winiwal  quality  levels  and  there  is  i»  regular  procedural 
device  for  encouraging  superior  qualify. 
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Federal  specifications  say  have  positive  qaalltles  such  as: 

providing  a basis  for  standardizatloa  - > • 

^ -**•*•• 

• establishing  Klnlaui  quall^r  levels 

• supporting  apparent  conpetitive  procarencnts  i . 

However,  the  CGOP  noted  that: 

’ *If  specifications  are  obsolete,  nany  conaerclal  products  do  not  aeet 
their  requlreaents.  This  In  effect  Units  conpetitlon,  defeats  the 
Intent  of  the  govemnent,  and  deprives  It  of  the  advantage  of  the 
technologically  dynuilc  open  ■arhet.*^ 

VALID  PURCHASE  DESCRIPTIOHS 


' r / . — 

..f  '•*  ’ ‘ ' -x  1 . 

.•  •1'’  .*-1^ 
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A prerequisite  for  the  use  of  advertised  procurenent  Is  the  establlshnent  of  a basis  for 
full  and  fair  conpetltlve  bidding  to  a cohboo  standard.  Procedures  surrounding  the  use  of 
advertised  procurenent  are  designed  to  preclude  favorltlsn,  collusion  or  fraud. 


The  principal  device  to  establish  a 
specifications. 


standard  has  been  the  use  of  formal  govemnent 


Thus,  any  effort  to  reform  current  advertising  procedures  nust  consider  what  constitutes 
an  acceptable  bid  purchase  description  In  lien  of  a formal  specification. 

In  order  to  be  valid,  purchase  descriptions  nust: 

•*  r , * ^ 

- accurately  reflect  the  needs  of  the  goverranent 

- avoid  unduly  restrictive  requlrenents  which  tend  to  Unit  ccnpetitlon  without 
satisfying  a real  need^ 

Current  ASPR  standards  derive  from  a long  history  of  GAO  decisions  relating  to  *brand- 
nane-or-equal*  procurenents.  Involving  the  use  of  a particular  type  of  purchase  description. 
This  history,  together,  with  related  GAO  decisions  and  the  regulatory  responses  which  followed 
these  decisions,  are  the  precedents  reiled  on  to  evaluate  what  constitutes  a valid  use  of 
functional  purchase  descriptions. 

It  Is  relevant  to  note  that  functional,  or  "performance  oriented  specifications*  have  been 
cited  by  GAO  as  best  protecting  the  government's  interest  In  obtaining  mlnlui  needs  at 
"favorable  prices”  while  Insuring  that  all  nu&facturers  have  *an  opportunity  to  compete  for 
government  purchases. 

Responsibility  for  Establishing  Functional  Purchase  Descriptions 

Ultimately,  the  government  must  perceive  and  determine  Its  needs,  especially  considering 
the  "second  guessing"  that  could  attend  the  use  of  a functional  purchase  description  approach 
to  procurement. 

"In  the  orderly  conduct  of  Its  business,  the  govemnent  as  a buyer  may  not  be  placed 
In  the  position  of  having  to  share  such  discretionary  authority  to  prescribe  Its  needs  with 
one  of  Its  potential  suppliers  who  Is  unable  to  adapt  Its  product  to  the  needs  of  the 
government. 
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While  the  Qovennent  nust  ultimately  retain  control  over  the  determination  of  Its  needs 
the  use  of  functional  purchase  descriptions  nay  Invite  a prellninary  round  of  "coiiwtltlon''’ 
for  Ideas  as  to  what  the  gover?»ent  ought  to  specify  as  nandatory  or  desirable  functions  to 
be  obtained.  In  connerclal  areas  of  rapid  technology  change,  such  as  autonatic  data  process- 
ing functions,  there  Is  a distinct  advantage  to  a user  to  search.  In  various  ways,  to 
becooe  atMre  of  what  functional  needs  can  be  net  by  the  connerclal  nartet-place 

Here,  the  user  Is  relying  on  the  capabilities  of  existing  cowrclal  eqidpmt  to  set 
functional  requirenents.  The  goverment's  "specification  workload*  has  been  asswd  by  the 
normal  activities  of  the  private  sector.  Potential  suppliers  response  to.  or  anticipation  of, 
^ of  what  Is  possible.  This  Is  the  essence  of  rellaKe  on  the  pHvate 

sector.*^  Moreover,  governnent  needs  could  be  discussed  In  pre-sol  Id  Ut  Ion  conferences, 
open  to  advocacy  and  challenge  by  potential  bidders.  Some  form  of  a nvidatory  pre-bid 
conference,  or  Its  equivalent,  probably  should  be  used  to  assure  a cooprehenslve  consideration 
of  existing  technology. 

Source  Limitation  Situations  - - ^ . 

Having  been  made  aware  of  private  sector  functional  capabilities  In  evolving  agency  needs, 
the  governnent  may  face  certain  source  llnltation  realities  In  meeting  govenment  needs  with 
existing  private  sector  equipment. 

The  basis  for  resolving  Issues  surrounding  source  limitation  concerns  has  been  an  "actual 
needs"  standard.  For  example,  sole-source  muards  are  acceptable  If  necessary  to  meet  "actual 
needs." 

Since  esUbllshment  of  specifications  reflecting  actual  needs  Is  the  responsibility  of 
the  procuring  agency,  the  government  has  not  violated  either  the  letter  or  spirit  of  foimml 
advertising  statute  In  the  past  because  only  one  bidder  could  supply  Its  ne^.  provided 
the  specifications  were  reasonable  and  necessary  for  the  purpose  Intended.^^  This  iom 
standard  should  be  available  for  functional  purchase  Ascriptions. 

Patents,  as  another  example,  may  be  Involved  In  equipment  necessary  to  satisfy  a*func- 
tlonal  need.  GAO  decisions  consistently  have  held  that  It  Is  permissible  to  specify  a 
patented  device  *.  . .so  long  as  that  patented  device  Is.  In  fact,  representative  of  the 
actual  needs  of  the  purchasing  agency. Thus,  a given  functional  purchase  description 
might  Involve  a need  met  only  by  a patented  Item. 

Stipulations  regarding  the  Independent  testing  of  equipment  have  also  been  permissible. 
There  was  concern  over  a Comptroller  General  decision  which  held  that  a requirement  that 
articles  supplied  to  the  government  bear  the  Underwriters'  Laboratories  label,  or  aqy  similar 
emblem  of  a private  organization,  "is  not  generally  justified."*^ 

Subsequently.  It  was  argued  that  agencies  of  the  federal  goverrmient  should  be  permitted 
to  utilize  the  recognized  good  management  procedure  adopted  and  followed  by  private  Industry. 
Finally.  GAO  backed  off.  "we  will  Interpose  no  objection.  . .If  It  be  determined  administra- 
tively In  each  case  that  such  a provision  Is  In  the  Interest  of  the  United  States*.!^ 

GAO  felt  that  "In  certain  cases  It  might  be  proper  to  Include.  . . a requirement  that  such 
articles  shall  conform  to  standards  adopted  by  recognized  private  testing  organizations 
where  such  standards  are  generally  recAnized  and  accepted  in  the  industries  Involved  and 
pertinent  to  the  government's  needs. ***^ 

From  the  precedent  of  latitude  to  state  actual  needs  with  restrictive  conditions  in 
formal  s^iflcatlons,  so  functional  purchase  descriptions  should  reflect  the  sa«  degree 
of  reality  In  meeting  actual  needs.  ^ 
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■Salient  fe<tures*of  Tunctloml  Purchase  Desc»^1pt1on$ 
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A partfount  cons  1 (Oration  In  the  use  of  functional  purchase  descHptions  Is  to  establish 
salient  functional  characteHstIcs  for  equl^wnts.  This  nust  be  done  with  considerable 
forethought  and  restraint. 

A ra9ulatory  "test*  has  evolved  for  salient  characteristics  In  general.  In  response  to 
GAO  decision: 

*purchase  descriptions  should  set  forth  those  salient  physical,  functional,  or 
other  charac^lstlcs  of  the  referenced  product  which  are  essential  to  the  needs  of  the 
goverment.'lo  . r ^ 

Again,  the  use  of  functional  purchase  descriptions  should  be  guided  by  GAO's  rule  In 
"brand-naae-or-equal*  decisions  that:  ...  ^ 

*A  bidder  offering.  . .products  should  not  have  tc  guess  as  th^  essential  qualities 
of  the.  . .Itea.  • .they  are  entitled  to  be  advised  In  the  :nv1tat1on  of  the  particular 
features  or  (functional)  characteristics  of  the.  • .Item  tdilch  they  are  required  to  meet. 

An  Invitation  which  falls  to  list  all  the  characteristics  deemed  essential  or  llsU  charac- 
teristics which  are  not  essential.  Is  defcct1ve.”19  ^ 

' Thus,  functional  purchase  descriptions  should  reflect  essential  functional  characteristics 
or  user  cost  considerations.^ 

Mhlle  pre-solicitation  conferences  o*  other  techniques  nay  be  utilized  In  part  to  deteraine 
needs,  the  needs  nust  be  settled  In  advarce  of  the  Issuance  of  the  Invitation  for  bids  and 
such  actual  needs  should  be  set  forth  In  the  bid  as  salient  characteristics,  (functional 
requlrenents),^^  together  with  essential  considerations  relating  to  user  costs.  ^ 

Response  to  Purchase  Description  , . • - 

Now  a bidder  responds  to  a functional  purchase  description  will  detendne  whether  the  bid 
Is  even  considered. 

The  descriptive  literature  clause  In  Federal  Procurement  Regulations  (FPR)  and  Armed 
Forces  Procurement  Regulations  (ASPR)  provide  that  thb  "Contracting  Officer  shall  Insert.  . * 
deUlls  with  respect  to  significant  elements  such  as.  « .performance  characteristics.  . .or 
operation,  as  appropriate. "22 

In  responding  to  an  Invitation  employing  a.  • .purchase  description,  the  responsiveness  of 
a bid  should  depend  not  on  whether  the  bidder  believes  or  even  knows,  that  his  product  meets 
the  purchase  description,  but  whether  the  procuring  activity  can  so  determine  from  the  Infor- 
mation submitted  with  the  bid. 23  . .descriptive  data  requirement.  • .Is  a material  require- 

ment which  cannot  be  waived.  Nerely  quoting  back  the  description  In  the  specifications 
(functional  purchase  description)  without  exception  would  not  In  Itself  fulfill  the  descrip- 
tive data  requirement.  . . "24 


It  should  be  noted  that  the  fact  that  a particular  firm  may  be  unable  or  unwilling  to 
meet  the  requirements  of  a solicitation  Is  not  sufficient  to  warrant  a conclusion  that  the 
specification  (purchase  description)  Is  unduly  restrictive. 29 

A blanket  statement  by  a bidder  offering  to  meet  all  requirements  (purchase  description) 
does  not  substitute  or  compensate  for  Inadeouate  descriptive  data,  and  rejection  of  (such  a) 
bid  as  nonresponsive  Is  therefore  requlred.^b 

A bidder  Is  free  to  offer  a product  with  features  In  excess  of  those  deemed  salient,  and 
the  government's  acceptance  of  sucn  a bid  docs  not  thereby  retroactively  alter  the  list  of 
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salient  features. 27  Such  features  night  affect  user  cost  detemlnatlon  In  esUbllshIng 
"least  cost*  to  the  govcnaent  anong  the  bids  received. 28  ^ 


KHm  DETEWOMtTIOW  ALTEWMTIVES 

In  considering  responsiveness  and  costs,  sealed  bid  anards  could  be  based  on  one  of  the 
following: 

* ^ • • * 

1.  AMrd  self  contalnedg  that  Is.  leltlil  bid  price  Is  the  sole  staadird  for 
deteralnlng  anard.  the  traditional  advertised  procedure. 

2.  IFB  sets  forth  Mndatory  functloes  and  desirable  functions  or  characteristics. 

A ntrlx  of  dollar  values  assigned  to  desIrable'Tunctlons  is  sUted  In  the  IF!  guide  to  all 
bidders.  The  net  cost  lepact  of  desirable  features  offered  or  not  offered  as  a part  of  bid 
response  are  considered  In  the  detendnatlon  of  an  awird. 

3.  IFB  sets  forth  oandatory  and  desirable  functional  features  and  In  addition,  pro-  ' 
vides  for  user  cost  calculations  In  order  to  detemlne  "lease  cost*  to  the  tmytr  for  bids 
received.  hHard  Is  detemlned  fron  netting  three  figures:  bid  prices  for  nandatory  features; 
net  cost  Impact  of  desirable  features  offered  or  not  offered;  user  costs  for  each  bid.  based 
on  use  cost  considerations  set  forth  In  the  IFB. 

Because  functional  purchase  descriptions  should  Invite  a range  of  approaches,  anard  deter- 
■Inatlon  techniques  nust  link  dissinllar  responses. 

The  nost  conplex  fom  of  award  detenrinatlon  Is  one  that  calculates  "lease  cost"  for  the 
govemnent.  Least-cost*  1$  also  the  nost  realistic  aoproach  for  award  detenriaatlon  where  user 
costs  can  oe  Identified  and  attributed  to  equlpnent  characteristics. 

Examples  i - 

Following  are  exanples  suggesting  the  use  of  functional  purchase  descriptions  In  IFBs. 

These  exaiaples  are  based  on  autonatlc  data  processing  equlpnent  purchases.  The  reasons 
for  selecting  this  txpn  of  product  for  Illustration  are  the  following: 

- The  govemnent  Is  the  largest  buyer  of  AOPE  equlpnent  and  services. 

- There  are  nr  standard  Govemnent  procurenent  specifications  for  AOPE  equlpnent.^ 

- The  conputer  narket  Is  highly  conpetitlve.^ 

• Coiqxjter  technology  Is  highly  dynanlc. 

- AOPE  purchases  represent  sone  of  the  nost  tine  consuning  and  conplex  purchases  under 
existing  procurenent  procedures. 

- AOPE  procurenent  continues  to  attract  Innovative  procedural  treatnmt  at  both  state 
and  local  govemnent  levels  as  well  as  the  federal  govemirnt  level. 

- If  the  precedents  and  tests  for  adopting  functional  purchase  descriptions  can  be 
nade  for  AOPE  equlpnent.  sipple  analogies  to  wore  routine  equlpnent  purchases  should  nake 
then  candidates  for  a functional  purchase  description  approach. 
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Least  Cost  Source  StIiCttw  TdOilque  *2 
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IFB  Includes: 

Mandatory; 

Hardware,  software, 
services,  described  as 
functional  requirements 
necessary  to  meet  actual 
needs.  Functional  state- 
ments encourage  Innovation 
and  maximum  competition 
In  meeting  needs. 

Desirable; 

System  characteristics  to 
enhance  or  fulfill  some 
aspect  of  actual  need. 
Dollar  values  assigned 
to  desired  functional 
characterl sites  based 
on  calculated  worth  to 
agency,  I.e.  cost  avoid- 
ance of  additional  outside 
purchase,  In-hocse  perfor- 
mance costs  or  sacrificing 
some  aspect  of  functional 
capacity. 

User  Cost  Considerations; 

System  life  considerafions 

which  can  be  readily  Iso- 
lated and  calculated  for 
various  bidder  responses 

power,  space,  supplies, 
air-conditioning  staff) 


Bidder  Response: 


Award  Determination 
Considers: 


I ‘•■'v  I 

• ' . I 


Must  agree  to  meet 
mandatory  functional 
requirements  or  bid 
Is  eliminated  from 
further  consideration 


•:;>  •.  'Hi  ■ ' ■ • ' 

May  or  may  not  re- 
spond to  any  or  all 
of  desired  system 
charactersitics; 
may  or  may  not  charge 
for  any  response(s). 


Bid  costs  on  a full  system ' 
life  costing  analysis  basis 
(e.g.  5 years,  7 years,  or 
whatever  time  period  Is  i 

. specified.)  , 

. . • -iu.  - 54* ir  1 T ; I i'jy  ■ v 

' - :-5  \ , 

• - • •••  * j > 

Evaluation  of  response  to 
each  desired  characteristic:  ^ 


Notice  given  In  IFB 
as  to  these  user 
cost  factors  and 
bidder  response 

life  cost  Implica- 
tions. 


(1  Offered  at  no  charge. 

or  offered  at  charge  be- 

low predetermined  value.  ^ 

subtract  either  assigned 

dollar  value  or  difference. 

from  bid  price. 

(2  Hot  offered  or  offered  ‘ 

at  charge  exceeding  pre- 

determined value,  no  de- 

duction from  bid  price.' 

r . - 

..3 

Net  of  system  life  cost 

dollar  Impacts  In  bidder 

approach. 

technique  nets  three  calculations  to  determine  an'^award- 
I.  bid  prlc^  for  •nandatory  requirements’* 

Of  -desired  Items,  features  or  additional  system  require-  ' 
■ents  offered  (user  pre-determined  values  for  additional  functions)  ^ 

**l*f^l*'  based  on  the  proposed  system  having  the  "least  cost"  1 e ' 

c^inatio^f  bld  costs  for  hardware,  software  & services;  net  of  values  for  desirable 
fte«  offered  or  not  offered;  related  user  costs.  ? *.  , 

* least_ total  cost  evaluation  technique,  the  functions  of  bid  items  listed  as  ' 
evaluated"  A bidders*  ability  to  satisfy  mandatory  fuHltiw^s 
te^rmlned.  utilizing  pre-atard* survey  or  other  techniques  if  necessary  if  m 

f.ti. ..  th.t  mS  IS 

evaluated  fw  additional  functional  contribution  are  those  items  called  "desired 
bi^  “•Xch  are  functional  characteristics  souSt 

agency,  unless  they  are  too  costly.  These  "other  system  reouirements^"’ 
items  required  for  a user's  system,  but  are  items  which  might  oossiblv  he 

4^!!"!^  performed  in-house,  or  what's  K>st  likelv  dm*  ^ 

without  at  some  sacrifice  to  weting  full  user  lU^s. 

''"HI?  *'  * bidder  to  offer  the  "other  deslrad  lto«"  fodicotes  ' 

doiiHl!  J^**^*y  «^blishing  a dollar  value  for  all  "desired  items"  is  to  make 
^la^  the  conon  denominator  between  evaluated  items  and  determining  a syst«™1ife 

f Purehase  as  an  exaiqile.  while  it  might  be  easy 

to  say  that  a 115  nanosecond  add  time"  would  be  desirable  it  ^ 

impossible  to  determine  a realistic  dollar  riorth  ' •*' 

**»‘png  feature  is  being  considered.  For  .v  ' 

”5  nanosecono  add  time"  is  not  the  important  con- 

important  consideration  is  more  likely  to  be  the  ' 

total  time  required  to  run  a representative  program  (consisting 
a«Ws).  together  with  a number  of  other 
representative  programs,  since  that  time,  extended  by  an 

‘he  number  of 

ho^  per  ^th  required  by  each  system  bid.  However,  even  that 
the  OTO  to  which  a dollar  value  can  be  assigned,  is 

(«*'•  "»«“►»)  after  all  the  known  and  lepresented 
5“  ***"  processed.  The  time  remaining  is  the  time 

‘he  expansion  capability 
considering  the  nature  of  the  work  load,  the 
no  4-  probability  of  the  work  lo^d 

^nfnr  ?»«‘lahle  to  do  it,  the  probable  cost  of  equip- 

ment or  rented  service  necessary  to  do  the  additional  work  Iwd,  etc. 


) 
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Siallarly,  the  value  of  a software  program  offered  by  a bidder  can  be  determined  by 
considering  the  cost  that  can  be  avoided  if  the  using  agency  does  not  have  to  write  the 
program,  buy  it  elsewhere,  or  operate  without  it.  The  value  of  an  item  should  be  the 
lowest  cost  alternative  available  to  the  using  activity. 

*If  the  cost  of  manually  flow  charting  is  $45,000  and  the  cost 
of  writing  a flow  charting  program  would  be  $32,000,  but  a - 

flow  charting  program  could  be  bought  for  $15,000,  the  value 
of  the  item  is  $15,000.‘34 

Not  only  is  It  necessary  to  predetennine  the  ck)11ar  value  for  every  "desired  IteeT 
to  be  used  In  the  evaluation, 

"It  Is  also  necessary  to  understand  how  the  established  dollar 
value(s)  vary  with  tine  (Is  It  $X  per  month  or  a one-time 
expenditure  of  $Y  and.  If  so,  when  Is  that  expenditure  made?) 

With  capability  offered  (If  the  flow  charting  prooram  offered 

^ by  a vendor  is  not  as  powerful  as  the  one  desired)  would  you: 
not  consider  It;  or  use  It  and  do  some  conventional  flow  charting 
on  Y a*  X of  your  programs;  or  etc?  The  pre-establlshraent  of  these 
variables  Is  contained  In  a value  or  assessment  matrix.  A matrix 
should  be  established  for  every  "other  system  requirement*  used  In 
the  selection.  This  matrix  showing  the  time  value  of  all  ^s1  red 
Items  must  be  disclosed  to  tl^ vendors  to  help  them  bid.*^ 

The  least  total  cost  technique  considers  not  only  the  cost  of  the  mandatory  functions 
but  of  all  features  sought  by  the  using  agency.  1)  System  cost  Is  calculated  for 
mandatory  equipment,  software,  support,  etc.  2)  Determinations  are  made  for  "other 
system  requirements*  If  a.bld^r  does  provide  an  Item,  but  at  some  cost  which  Is  less 
than  Its  stated  value,  that  lesser  cost  Is  subtracted  from  his  bid.  If  no  charge,  then 
entire  va'ue  is  subtracted.  3)  Then  the  "user  costs"  for  expenses  like  power,  staffing,’ 
etc.,  are  calculated. 

In  this  manner  costs  of  meeting  functional  requlrenents  over  the  stated  system  life 
are  considered  In  the  evaluation.  The  system  having  the  least  total  cost  calculated  for 
the  prescribed  system  life  Is  the  system  that  Is  selected. 

The  Idealized  approach  presented  here  Is  based  on  the  procurement  of  a complete 
system  for  aspects  of  a given  system  life  cycle.  The  same  technique  would  apply  for 
acquisition  of  Individual  system  elements  such  as  plug-compatible  direct  access  devices 
or  for  the  acquisition  of  software  packages.  Even  with  Individual  system  components, 
there  are  considerations  In  addition  to  the  component  price.  Power  requirements  may 
differ,  site  prerequisites  could  cause  expenditures  over  and  above  the  amounts  required 
for  just  the  hardware.  Software  or  operating  system  changes  could  prove  to  be  costly 
and  thus  reduce  an  apparent  cost  advantage.  Such  costs  should  be  evaluated  over  the 
system  life  cycle  of  the  Individual  cenponent  or  software  package  and  considered  In 
determining  award. 

Bidder  strategy  In  responding  to  a "least  cost"  approach  would  be  to  determine  mar- 

?1nal  costs,  compare  to  potential  respwses  of  other  bidders  and  "rewards"  calculated 
n value  matrix.  In  this  way.  Innovation  and  a competitive  character  should  pervade 
bidder  decision-making. 
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Bidder  strategy  In  responding  to  a "least  cost"  approach  would  be  to  determine  mar- 
ginal costs,  compare  to  potential  responses  of  other  bidders  and  "rewards"  calculated  In 
value  matrix.  In  this  way.  Innovation  and  a competitive  character  should  pervade  bidder 
decision-making.  r 

The  objectivity  of  evaluation  criteria  for  user  costs  should  be  based  on  easily 
verifiable  characteristics,  e.g.  power,  staffing,  materials,  space,  etc.  The  value  for 
desired  characteristics  should  also  be  based  on  verifiable  ^aracterlstlcs,  e.g.  net 
time  to  accomplish  needs.  Any  weighting  among  characteristics  should  be  determined  In 
advance  and  nhde  a part  of  the  bid  solicitation  so  all  bidders  know  the  basis  for  award  ^ 
determination.  ^ ^ 


EIAHPLE  11  r • *r  ..-..-r  -/ 

Outline  of  ^rd  Determination  Considerations 

Objective:  Use  of  IFB  for  purchase/least  of  conputer  systen  and  cooponents,  support 
services,  aanuals,  Mintenance  services  on  a least  cost  basis.  •-> 


Issues  • ".-.f 

17  Establish  purchase  description  (called  systea  hankare  and  software 
’specification*)  . 

a.  Rake  needs  analysis  (this  Is  a disciplined  requirenents  detenainatlon 
. . . ...  > processTwe  procedures  for  wblcb  are  given  In  *conputer  procurenent 

' • gu1deI1nes*)3/  ^ 

b.  Establish  required  functional  capabilities  to  neet  needs  (part  of  the 
. ~ reoulrenents  detemi nation  process  acconp fished  by  In-house  technical 

staff,  utilizing  supplenental  resources  as  Indicated  below) 


User  Input:  a policy  advisory  conalttee,  coniwsed  of  the  prospective  najor 
users  of  the  systen,  review  the  required  capabilities  to  be  sure  that  ' 
they  net  out  ot  neet  the  potential  users  actual  needs. 


Dinent  data  processing  Installation  managers:  a review  coenlttee  of  managers 
of  data  processing  Installations  Is  selected  on  the  busis  that  their 
Installations  utilize  equipnent  manufactured  by  potential  bidders  on  the 
subject  IFB.  These  managers  have  a practical  working  experience  for 
reviewing  the  content  of  an  IFB,  especially  as  regards  source  selection 
criteria. 

• J 

Possible  variation;  a "rough-cut"  purchase  description  could  be  Issued  to 
Industry  as  the  basis  for  a pre-solicitation  conference;  refinements  to 
this  rough-cut  could  be  the ^result  of  exchanges  in  a pre-bid  conference; 
this  technique  could  contribute  to  a revised  needs  concept,  either 
Increasing  or  modifying  the  Initial  purchase  description. 

Basic  Premise:  reliance  on  the  private  sector  as  the  source  for  meeting 
government  needs  means  that  a "competitive  process"  is  implicitly 
underway  when  the  government  enters  the  cooinerclal  market  place.  One 
of  the  benefits  to  be  derived  by  the  government  from  this  on-going 
competition  is  to  measure  its  own  capabilities  and  understanding  against 
what  is  available  and  thus  tailor  government  functional  requirements 
determination  based  on  the  broadest  technology  base.  Functional  purchase 
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descriptions  should  be  written  with  minimum  constroints  to  elicit 
. maximum  competition.  On  the  other  hand,  there  is  no  reason  why  govern- 
ment must  seek  the  lowest  connon  denominator  of  "needs  statement"  just 
to  guarantee  a large  nimiber  of  bidders*  The  government  ought  to  be 
able  to  specify  its  functional  needs  based  on  a full  understanding  of 
what  exists  in  the  technology  base. 

2.  Maximizing  product  co^)etition  and  enhancing  bidder  understanding 

a.  Use  of  bidders  conference  - to  provide  clarification  of  items  contained  in 
IFB;  (could  utilize,  in  part,  to  describe  anticipated  variations  in 
coiputer  use  environment  to  elicit  response  or  observations  from 
prospective  bidders);  all  pre-bid  conference  proceedings  recorded, 
transcribed,  sunaarized  and  distributed  to  all  prospective  bidders. 


.U 

— ■ • 

■ u 
. O 

• i 


b.  Interpretations  - binding  on  govenaent  only  If  set  forth  In  writing; 

text  of  requests  and  Interpretations  sent  to  all  prospective  bidders. 

c.  Details  of  systea  - each  deliverable  component  listed  separately  In  IFB 

response  to  clarify  system  content. 

♦ 

d.  System  demonstration  requirements  of  IFB  give  government  option  to 

require  demonstration  to  verify  performance  or  compare  competing  systems 
during  bid  evaluation.  . 

• » • 1 .11.  ■ 

e.  (hiestions  section  - in  order  to  be  responsive,  bidder  mast  answer  an  extensive 

list  of  questions,  to  be  used  as  an  aid  in  evaluation  of  bids  (neither 
questions  nor  responses  are  used  in  contract  document). 

Creating  opportunity  for  innovation  in  bidder  responses 


u 
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\ 
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a.  Alternatives  to  system  hardware  and  software  purchase  description 

(specifications)  invited: 

Provided:  * " ' V ‘ 

Bidder  first  has  been  responsive  and  fully  complied  with  all 
requirements  in  basic  IFB 

Bidder  alternatives  not  subject  to  any  negotiation 

Alternatives  considered  in  determining  "best  bid"  for  purposes  of 
making  contract  award 

b.  Modifications  (Amendments  and/or  additions  to  bid  invited  for): 

Form  of  bid  (Price  & rate  schedules,  lease  w/option  to  purchase, 

performance  bonds,  etc.) 

Deliverable  items  and  performance  requirements  (Deliveries,  warranty, 
licensed  software,  equipment  maintenance  services,  etc.) 

Acceptable  provisions  (Demonstration  of  future  capabilities,  acceptance 
test  period,  standard  of  performance,  etc.) 

General  contract  provisions  (Definitions,  insurance,  equal  opportunity, 
rights  in  data,  taxes,  invoices,  and  payments,  risk  of  loss  or 
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v*  danage,  survival  of  terns  beyond  expiration  of  agreenent,  etc.) 

Provided:  ^ ^ ^ ^ u_  ..  ^ u: 

• ^ ‘ ' ■ •*J  . v'*-  ' 

Bidder  Bust  designation  nodificatlons  as; 

"required" 

1^*  tr.  ••;rr  ;r.^  f t r^.sf  j Vj*Si 

W ‘M;r'*o»*  »•  • i.if.  t.fc-*, -i*  .••:c!  r>  i::  • 'iu  vh-oi 

'.^-1^.^  ’ ■•  f.v:;  <-1  .‘-.-f'?,  .:;  pru  . 

..■•'^•*■>^1'**''^  . • -‘i  t::-  •<:•■;  • * i ;?jvy>V^f 

. . « i - 

Bidder  aust  indicate  cost  impact  for  buyer  excliidinq  any  bidder 
proposed  revision  . . . . . , ..iM 

' 

Bqyer  reserves  the  right  to  accept  any  or  all  nodificatlons 


4.  Oetemining  evaluation  and  award  factors 


Jrs'iH 

r-y-r-r, 


a.  Basis  of  determining  costs:  Total  costs  to  procure,  install  and  operate 
5..,^*  , .*  - system  (calculated  for  a seven  year  period) 

J‘»  ..  ^ r • **  ..*  -St  >\  >r.  ♦* 

Including:  ^ ^ 


A •>  n*/  ^ . Pi. 


Purchase  price 
Naintenance  charges 
License  charges,  software 
Naintenance  fees,  software 
Stental,  prograa  product 


Technical  support  services 
Nanuals 

Nonthly  lease  rentals 
Conversion  costs 
Training 


Any  ayitlonal  cost  adjustment  to  Insure  that  total  costs  to  utilize 
^ ' the  systen  for  a 7 year  period  are  considered 

Basis  of  award;  Najor  consideration  given  to  lowest  total  cost  to  utilize 
hardware  & software 

Other  factors  (desirable  characteristics)  of  najor  lB|)ortance: 

Interface,  exchange,  flexibility  considerations;  efficiency  A 
sinpllcity  of  data  nanagement  concept;  ability  to  expand  for 
out-year  applications. 

Possible  variation:  a natrlx  of  values  for  desirable  characteristics 
could  be  established.  Including  associated  dollars,  expressed  in 
the  IFB  as  detailed  in  least  cost  example  above;  other  Itesis  could 
be  Included  In  a "question  section";  togetlier  these  would  eliminate 
or  nininize  controversies  over  undifferentiated  "najor  considerations", 
other  than  bid  price,  being  used  as  evaluation  criteria  and  would 
^ have  the  effect  of  converting  qualitative  considerations  into 
neasurable  dollar  differences. 


231 


( ) 


.i/C 


j 


CONCLUSION 
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The  rigidity  ano  costs  associated  Mith  for^l  Govemment  specifications  nay  not  serve 
the  best  Interests  of  the  Govenoent  froe  the  standpoint  of  aeetlng  actual  user  needs  or 
achieving  louest  ultlnate  cost.  j <.^ 


This  paper  has  suggested  one  conceptual  alternative  to  such  problens  through  the 
use  of  functional  purchase  descriptions  and  auard  determination  procedures  which  Involve 
qualitative  judgements  and  user  cost  considerations. 


Award  determinations  based  on  Innovative,  competitive  proposals  wd  their  related 
cost  1^>11cat1ons.  argue  for  a new  way  of  looking  at  formal  advertising.  This 
standard  Is  ultimate  costs  related  to  a product  and  Its  characteristics  rather  than  a 
narrow  concept  of  Initial  bid  price  as  the  basis  for  ward. 


S1^>1y  stated,  this  1$  product  competition  rather  than  Initial  bid  price  competition 


('■) 


Use  of  functional  purchase  descriptions  Is  only  relevant  to  situations  where  there 
Is  a strong  privata  sector  technology  base  to  serve  particular  govenwent  needs. 


The  references  cited  represent  conplex  and  tlie-consunlng  exaiples  of  sooie  of  the 
basic  principles  Involved  In  the  use  of  functional  purchase  descriptions.  The  purpose 
of  citing  such  cois>1ex  examples  is  to  require  coKlderatlon  of  the  maxlmmi  nuaber  of 
variables.  The  basic  principles  should  survive  these  complex  examples  for  routine  use 
In  much  simpler  procurements. 


^D^lte  the  complexity  of  the  AOPE  examples,  the  buyer  was  In  a position  to  evaluate 
• "wher  of  competing  and  Innovative  approaches.  Options  were  afforded  to  bidders  which 
In  turn  gave  the  buyer  a qualitative  choice  In  determining  which  cooblnatlon  of  factors 
best  met  actual  needs  for  the  lowest  ultimate  cost. 


The  past  history  of  Comptroller  General  decisions  on  related  purchase  description 
Issues  has  established  concepu  of  valid  purchase  descHptlon,  actual  needs  and  salient 
characteristics,  all  of  idiich  appear  to  be  adapUble  to  guide  the  use  of  functional 
purchase  descriptions  In  sealed  bid  proceedings. 


It  appears  that  the  govemment  often  may  avoid  the  costs  of  developing  and  adminis 
tering  fonnal  govemment  specifications  for  many  products,  while  affording  Itself  the 
opportunity  of  considering  the  latest  technology  and  Innovative  approaches  In  meeting 
govemment  needs  at  the  lowest  possible  cost. 
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Footnotes 


Federal  Acquisition  ."ct  of  1977.  Senate  Bill  S.1264;  95th  Congress 


1st  Session 


Comp.  Gen.  Dec.  17:554:560,  Jan.  8,  1938;  C.G.a  B-157053.  Aug.  2. 

1965;  C.G.O.  B-169140,  July  8,  1970;  as  cited  in  Report  of  Coomlsslon 
on  Government  Procurement  (COGP),  vol.  3,  p.  15. 

CXP  vol.  3,  p.  15.  . ■ ■ • 

• t 1' -K' • . » j,’  '*  £\i 

JBi^,  p.  16.  _ ••  v;.  c V'sii: 

See  Government  Contract  Bidding  by  Paul  A.  Shnitzer.  Associate  General 
Counsel,  GAO;  Federal  Publications.  1977;  The  definitive  mork  on 
^ bidding  concerns  including  specifications. 


Ibid,  p.  20.  - ' : 

Ibid,  p.  20  ) • ..  -f.v  3i 

• ' •'  S-  * ’ • :»  rr».,  r-  vt  pA-JO 

ASPR  1-1206.1.  • " ■ • ■ ' • 

."•■VI  •-IT 

C.6.0.  B-161704.  Sept.  26,  1967.  v r*- 

: • -t  -j-Ki 

C.G.O.  B-162475,  Har.  19,  1968.  J - • 

The  policy  for  reliance  on  the  private  sector.  CHS  Circular  A-76,  is 
currently  under  review  by  the  Office  of  Federal  Procurement  Policy. 

34  Comp.  Gen.  336;  45  id.  365  (cited  in  C.G.O.  B-162475.  Mar.  19.  1968) 

C.G.D.  B-161118.  June  1.  1967,  . ^ 

33  Comp.  Gen.  573.  ' . . , ^ .J 

C.G.O.  B-116236,  Oec.  5,  1955. 


41  Comp.  Gen.  242.250-251.  See  also  C.G.D.  8-157857,  June  26,  1966 
cited  In  48  Comp.  Gen.  441  (1968);  49  Comp.  Gen.  247  (1969); 
C.G.D.  B-173290,  Oct.  19,  1971;  referenced  in  C.G.O.  8-175955, 
July  25,  1972. 


Sample  extracts  of  Purchase  Description: 


SYSTEM  HARDWARE  AND  SOFTWARE  SPECIFICATIOftS 

7.1  yVERALL  SYSTEM  REQUIREMENTS 


7.1.1 


General  Capabilities 

The  proposed  computer  System  must  Include  the  necessary 
HardMre.  Software  and  related  support  services,  which  have 
the  capability  to  meet  the  requirements  of  this  Agreement 
and  to  perform  the  following  types  of  processing  concurrently 
as  they  are  defined  In  Section  11.1: 

A.  Timesharing;  ' . 

B.  On-Line  Transactions  Processing;  i ' i - 1 

C.  Remote  Batch  Job  Entry;  and  , . ^ «i:^'  sa' 

0.  Local  Batch  Processing.  I 


7.1.2  Physical  Limitations  i'; 

' ' " The  computer  System  with  peripheral  equipment  as  Initially 
contracted  for  «ist  operate  within  the  following  physical 
' * 11m1t^i«ons:  PHYSICAL  LIMITATIONS:  an  area  21  feet  by 

36  feet.  The  layout  of  equipment  In  the  21*  x 36*  area  must 
allow  sufficient  space  for  all  necessary  maintenance  and 
operation.  If  additional  space  Is  required.- this  wst  be 
clearly  stated  In  the  Bid  under  this  section.  ... 


7.1.8  Privacy  and  Security  -.The  Bidder  should  specify  the  security 
artd  privacy  capabilities  of  the  computer  System  In  the  modes 
Indicated  In  Subsection  7.1.1.  It  Is  expected  that  these 
capabilities  will  be  reasonable  In  light  of  the  state  of  the 
art  and  the  protection  provided  to  the  right  of  privacy  as 
specified  In  the  California  Constitution.  ... 
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2‘  C.G.D.  B-165409,  Dec.  26,  1968. 

22  ASPR  2-202.5  (d)  (i);  FPR  1-1.307-4. 
C.G.D.  B-161122.  May  11,  1967. 

41  CoDip,  Gen*  366 

^5.  30  Comp.  Gen.  368  ' 
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26|  45  Co«p.  Gen  312,  316  (1965);  41  Id.  366  (1961);  cited  In  C.G.D. 
’ B-169482,  Sept.  16.  1970.  ” 

>.■* ' “'■•^'^"7  ■f‘wn6  ‘ , 
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27 

28 


C.6.0.  B-1 77229.  Feb.  8.  1973. 


“Least  Cost”  approach  will  be  amplified  in  examples  below.  It  involves 
the  quantification  and  consideration  of  anticipated  costs  as  well  as 
bid  costs.  - _ . -•  ‘ , 

29  C06P  vol.  3.'p.  45.  •'  * - - r . 

Ibid.  . ' 

2^  Models  used  in  this  paper  are  based  on  Computer  Hardware  Procurement ' and 
. Contracting  Guidelines  of  the  Intergovermental  Board  of  Electronic 
Data  Processing,  1025  P Street.  Sacramento,  CA  95814. 

The  terminology  "least  cost"  and  the  example  are  modeled  on  the  approach 
given  in  the  Guidelines  cited  in  31  above.  ' 

Guidelines,  p.  37. 

9**  Guidelines,  p.  38. 

2®  Guidelines,  p.  38.  " 


32 


( - 


2®  This  case  was  supplied  by  Central  Valley  Educational  Data  Processing 
Center,  Fresno,  CA  93721. 


22  Cited  in  31  above. 
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THE  FOUR  STEP  SOmttE ! 


1 ' 


Lie  Douglas  C.  Dillon.  USAF 
Staff  Procureaent  Specialist 
QASOCMMl) 


I ^ 


:i 


. .w>f nyJ  j ' r;^ 

; :.^.;,.V;^  :tr..-t!2 


LK  Dillon  spoke extcrporaneously  on  his  subject  and  did  not  suMt  a paper  per  se.  The 


folloifing  abstract  and  charts  should  give  an  Idea  of  his  talk. 


'Hs* 


U-^>. 


The  revised  000  Directive  4105.62  dated  6 January  1976  Is  the  top  000  document  on 
selection  of  contractual  sources  for  major  defense  systems;  It  covers,  among  other  things, 
the  source  selection  authority  responsibilities  and  organization,  solicitation  preparation.' 
and  evaluation.  A major  concept  Introduced  here  is  the  4-step  source  selection  proce<hires 
and  Its  objectives.  An  evaluation  of  the  4-step  test  program  has  recently  been  completed 
nts  on  preliminary  test  results  and  trends  can  now  be  made. 


(See  attached  charts  and  explanations) 
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This  chart  reflects  assessment  areas  being  used  by  the  DoO 
Evaluation  Group.  Of  particular  importance  in  the  second  item 
is  the  word  "may".  It  is  nearly  inqwssible  to  say  with  any 
degree  of  confidence  that  technical  leveling,  auctioning  and 
buy-ins  actually  occurred  on  any  given  program.  We  do  however 
hope  to  identify  conditions  or  circumstances  that  may  lead  to  or 
Indicate  these  things  could  happen. 


/CESSMENT  AREAS 


r 


IDENTIFY  METHODS  AND  PROCEDURES  TO 
IMPROVE  SOURCE  SELECTION  PROCESS 

IDENTIFY  CONDITIONS  AND  CIRCUMSTANCES 
THAT  MAY  LEAD  TO  OR  INDICATE: 

II  TECHNICAL  LEVELING 
2)  AUCTIONING 
3}  BUY-INS 
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The  Initial  DoO  Directive  4105.62  was  Issued  In  1965.  Around 
1970  then  SecDef  Packard  was  concerned  about  how  we  select  our 
contractors  for  oajor  weapon  systens.  One  of  the  results  of  his 
concerns  was  the  Issuance  of  DoO  Directive  5000.1.  With  the 
Issuance  of  this  directive*  It  was  recognized  that  OoD  4105.62 
required  revision  and  updating. 

Also  during ‘this  tine  there  was  considerable  eaphasis  within 

' r -;.T  , - f'  .t  ■ « 

the  DoD  to  Inpleaent  the  4*step  procedures.  It  was  decided  by 


the  OoD  that  with  the  Issuance  of  the  new  Directive  4105.62 

O'' 

dated  6 Jan  1976  the  new  4>step  procedure  would  be  tested.  It  , , 


was  thought  by  soae  quarters  that  the  proposed  4*step  procedure 
would  alleviate  these  assertions  by  Industry  and  the  government. 
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This  chart  shows  the  criteria  currently  being  used  by  the  OoC 


’ -.'-T 


Evaluation  Group  against  which  prograa  data  Is  neasured.  Notice 
the  Mjor  headings  correspond  to  a previous  chart  entitled 

Test  Pur'pose.  "’  ' - < sf-  isw 

liA  JS.4*  t;;4j  crw  Si  r 7rft* 

1 ■ ’ - - i * 

CRITEIUA  - ; 

IMPROVE  SOURCE  SELECTION  TECHNICAL  LEVELING 


' ) 


\J  \i^r 
0 


v/n  i ; 

SOLICITATION  QUALITY 


OEHCIENCY  DISCUSSIONS 
MULTIPLE  SCORING 


PROPOSAL  QUALITY 
PERSONNEL  UTILIZATION 
SOLE  ^URCE/MULTIPLE  NEGOTIATION 
PROTEST  ACTIVITY 

AUCTIONING 


' ' REPETITIVE  SCORING 
- RFP  AMENDMENTS  > 


BEST  AND  FINAL  OFFERS 
HPA  WAIVERS 


BUY-INS 


COST  ESTIMATES 
PROPOSAL  REOUCnOPJS 
PROPOSAL  INCREASES 
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This  chart  depicts  those  programs  currently  Identified  as  4-step  test 
programs,  and  the  development  phase  each  program  is  currently 
experiencing. 


TESTPROGRAMS 


DEVELOPMENT  PHASES 


EXPLORATORY 


• JTIDS 

• ENGINE.  nOHP 


I ADVANCED  ( EMWPMiWG 

• •SUB/AOI  COMM  SY8TBM  • SOFT  RAG 


•FURSaiSOR 

•ECHO  RANGE 
TRANSMITTER 

•MARK/GUN  LASER 
•ASWPODS 

•nis  . .. 


• ROCKET  LAUNCHER 

• LAMPS-MKM 

• SPS-19  RADAR 

• SAWS 

• BSZ/KC-ISS  SUWULATOa 

• SATIN  IV 

• JSS 

• mVERTER-SKW 


This  chart  represents  the  4-step  process  as  currently  being  tested. 
Notice  that  the  lower  step  of  the  current  process  has  now  been 


.broken  out  Into  two  steps. 


• c ’ 


FOUR  STEP  PROCESS 


NEGOTIATION  OF  CONTRACT 


EVALUATION  OF  TOTAL 
PROPOSAL  AND  SELECTION 
OF  APPARENT  WINNER 


RECEIPT  AND  DISCUSSION  OF 
COST/PRICE  PROPOSALS 


RECEIPT  AND  DISCUSSION  OF 
TECHNICAL  PROPOSAL 
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As  a natter  of  reference,  this  chart  depicts  the  current  procedure 
used  by  the  OoO  In  selecting  Its  contractual  sources. 

* ’i*’*  1 

-“•(1  •i'-i  i '"A  •'••-■d  : O rl  ■J.i'  - t'* 

.'I-'.*;?  £:••••«'*»  A.’'  i— ‘".f  r.%  .- 


CURRENT  PROCESS 

“■  . r ..  .'4  I - ■ 


NEGOTIATION  OF  CONTRACTS 
AND  SELECTION  OF  WINNER 

EVALUATION  OF  PROPOSALS 


RECEIPT  OF  TECHNICAL  AND  COST 
PROPOSALS 


The  purpose  of  the  4-step  test  program  is  to  Improve  the  quality 
of  our  overall  source  selection  process  and  to  eliminate  or  reduce 
program,  technical  leveling,  buy-ins  and  auctioning.  There  have 
been  accusations  by  industry  that  the  government  Indulged  in 
technical  leveling  and  auctioning  during  the  source  selection 

process.  Similarly,  there  are  accusations  on  the  part  of  the 

<« 

government  that  contractors  buy-in. 


TEST  PURPOSE 


• IMPROVE  QUALITY  OF  SOURCE 

SELECTION  PROCESS 

• TO  ELIMINATE  OR  REDUCE 

PROGRAM 

• TECHNICAL  LEVELING 

• BUY-INS 

• AUCTIONING 


t 
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% 


The  source  selection  authority  responsibilities  remain  as 
shown  on  this  chart.  These  responsibilities  have  not  changed 
as  a result  of  the  new  DoO  directive. 


i ■ 


S.S.  AUTHORITY 

f - 

,n  - . « >V  1.  ..  t »..*  it.  (.i  ‘..-r  ^ 


9 COE^i^UCT  PS^OCESS^ 
e SELECT  SOUSiCES 
9 DCP/SECOEF  EViEE^O 
9 SELECTEOE^  RESULTS 
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The  new  OoO  Directive  4105.62  does  sake  so*  changes  to  the 

source  selection  process.  However,  there  are  some  things  that 

renain  unchanged  as  shown  on  this  chart.  . [ 

I 

I 

UNCHANGED  ASPECTS 

■ . ■ • , 5 

"A  '/  • .« 

f ^ 

e SOURCE  SELECTION  OBJECTIVES 

• SOURCE  SELECTION  AUTHORITY 

• SOURCE  SELECTION  ORGANIZATION 
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As  we  envision  at  the  current  tine,  the  test  results  will 
dictaU  a decision  on  the  part  of  the  DepSecDef  as  represented 
by  this  chart. 


T£STR£SUL7S 


USE  WHEN  APPROPRIATE 


DO  NOT  USE 


MAKE  APPROPRIATE  CHANGES 
TO  DIRECTIVES 
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This  chart  depicts  the  vertical  organization  for  testing  the 
4- step  process.  Data  Is  received  from  the  three  Service  components 
by  the  Department  of  Defense  Evaluation  Group.  This  Evaluation  Group 
Is  the  working  organization  actually  reviewing  the  data. 


The  OoD  Steering  Group  Is  an  advisory  body  to  the  OoD  Evaluation  Group. 

m 

In  addition,  at  the  completion  of  the  test,  they  will  make  a recommendation 
on  the  adoption  or  rejection  of  the  4*step  to  the  DepSecOef.  His 
decision  will  result  In  DoO  policy. 


EVALUATION  PROCESS 


EVALUATION 

V.^ROUP^ 


DOD 

STEERING 

^GROUP_ 


SEC  DE^ 
V^CISION^ 


POUCY 


L 1.1  uLi.  I 


This  chart  represents  the  various  types  of  program  included  in 
the  4-step  test.  . ■ 

- . »•  c.-r.  r,.  r.?  .14,  ijij  ^ 


TYPES  OF  PROGMMS 


SPACE/MISSILES 

/ 


This  chart  represents  those  organizations  participating  In  the 
test  of  the  4-step  source  selerMon  process.  The  tern  contractors 
represents  those  contractors  who  are  participants  In  one  of  the 
4-step  candidate  programs.  COOSIA  Is  an  organization  whereby  those 
firms  who  did  not  participate  In  a test  program  due  to  product  line 
or  some  other  reason  uy  have  a means  to  make  an  Input  to  the  evaluation 
of  the  4-step  process.  Near  the  end  of  the  test,  COOSIA  will  submit 
a position  paper  to  the  OoO  Evaluation  Group. 


CONTRIBUTING 

ORGANIZATIONS 
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DOD 
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OolUr  values  of  test  prograas  range  fro«  $100,000  to  over  " ’ 

$1  billion  represented  by  the  dotted  line.  The  wide  range  of  ' ''  •, 

'prograa  values. is  proving  to  be  very  beneficial.  ' ‘ 

? . . j.*,  * <3.»  M l.o**  '• 
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PROGRAMS  DOLLAR  RANGE 
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SOURCE  SELECTIM  RESEARCH  - IS  A?<YBOOT  HffEWtSTEB? 


LtCol  F.  Theodore  Helner 
Associate  Professor  of  Manageoent 

OS  Air  Force  Academy  t . ^ ■\ 

“•  ■ * • 

...  • • ..  -o*  ; ; • 

Before  I begin  this  discussion,  let  ne  say  that  I solicit  your  feedback  and  reactions 
to  cooRnts.  brief  reaarks  will  further  enphaslze  the  need  for  translating  procure- 
■ent  research  Into  action— the  wajor  theme  of  this  sy^wslun.  My  sincere  concern  for 
translating  research  results  Into  application  Is  the  sole  reason  I accepted  Bob  Williams* 
Invitation  to  address  this  distinguished  group.  I want  to  share  with  you  ^y  frustrations 
as  a procurement  researcher  In  getting  the  attention  of  the  "system"  In  wy  attempts  to 
make  change  rather  than  to  Just  publish  a research  paper  for  academic  and  officer 
performance  report  credit.  ^ . 

In  particular,  I will  discuss  two  source  selection  research  projects  that  have  caused 
this  frustration.  1 hope  the  output  of  this  symposlimi  will  help  minimize  this  frustration 
for  future  researchers  In  this  area.  After  hearing  General  Stansberry  yesterday,  I have 
begun  to  wonder  If  ^y  two  experiences  fall  Into  the  category  of  non-problems.  If  so,  you 
may  disregard  my  remarks  totally  and  enjoy  the  view.  If  not,  then  I hope  that  we  all 
leave  West  Point  with  a greater  knowledge  of  how  research  should  have  some  Impact  upon 
the  system. 

Let's  get  back  to  the  topic  of  source  selection  research.  Last,  year,  this  group 
honored  Kajor  Bob  Taylor  and  me  with  one  of  the  NCMA  awards  for  our  paper,  "A  Conceputal 
Model  for  Management  Evaluation  During  Source  Selection."  We  have  printed  over  400  copies 
of  this  report  and  have  about  25  left.  We  made  an  Initial  distribution  of  about  100  to 
libraries,  buying  agencies,  OLSIE,  etc.,  throughout  the  Department  of  Defense.  Upon 
request,  I recently  presented  that  paper  at  the  National  Convention  of  the  American 
Institute  of  Industrial  Engineers.  I had  a room  capacity  crowd.  Perhaps  more  signifi- 
cant, however.  Is  the  fact  that  we  mailed  out  over  200  copies  at  the  specific  telephone  or 
written  request  of  people  In  the  field  who  are  Involved  In  source  selection  In  several 
governmental  agencies  and  In  subcontractor  selection.  The  positive  response  of  most  of 
these  requestors  has  been  the  same— they  consistently  ask,  "Why  didn't  I learn  of  your 
research  sooner?"  (I.e.,  before  we  sent  out  our  RFP)  and,  "When  will  DOO  react  to  your 
Ideas  and  modify  source  selection  procedures?"  You  can  readily  see  our  frustratiur.— 
we  can't  answer  these  questions.  I don't  mean  to  Imply  that  our  research  Is  the  greatest 
thing  since  sliced  bread,  but  It  does  suggest  a different  way  of  attacking  one  aspect 
of  source  selections,  and  the  feedback  from  procurement  practitioners  has  been  over- 
whelmingly positive.  We  have  received  no  response  from  higher  headquarters,  except  one  of 
our  telephone  Inquiries  was  answered  by  the  coeoent  that  "we  tried  that  once  a long  time 
ago  and  It  won't  work." 

Perhaps  some  Insight  Into  the  origins  of  this  research  will  help  you  understand  our 
frustration.  About  two  years  ago,  I was  asked  to  head  the  management  panel  during  the 
source  selection  of  a major  satellite  program.  I arrived  In  Los  Angeles  about  the  time 
the  proposals  did,  formed  a team,  and  started  comparing  apples,  oranges,  bananas  and 
peaches.  There  was  nothing  In  the  literature  or  000  publications  to  help  us,  and  I was  so 
frustrated  by  this  experience  and  the  shallowness  of  ny  recomnendatlons  that  I began  this 
research  topic  with  Bob  Taylor.  We  visited  numerous  DOO  contractors  to  learn  how  they 
evaluated  their  major  subcontractors  during  source  selection.  This  Input,  plus  Interviews 
with  meiabers  of  previous  Air  Force  source  selections,  resulted  In  our  research  report. 


I sincerely  telleve  we  have  sons  good  ideas,  yet  the  ‘systea*  seens  to  ignore  our  efforts 

I"*"  *««"  «»■  »"’y  *«i  doer 

?***  suggestions  are  being  used.  Please  reoieober  that  this  research 
was  total  unsponsored— ic  just  saw  the  need  to  improve  and  tried  to  make  a contribution. 


Hy  second  att^t  at  unsolicited  source  selection  research  cane  as  the  result  of  sow 
mearch  I was  doing  with  ^e  RANO  Corporation.  I was  working  on  the  optimal  future  utilize- 

awl  we  developed  a complex  computerized  O.R. 

» I*"!^**®  "I"!*!*!  ]<f«-cycle  costs  and  maximize  mission  effectiveness.  As  it 

*****]  immediate  application  in  the  evaluation  of  several  alternative 
selection.  With  the  input  of  only  (1)  the  mission  being 
Sir  fc!r  ***^  parf^nce  ^racteristics  of  each  aircraft,  and  (3)  the  life-cycle  costs 

hour,  our  model  would  indicate  which  of  several  considered  aircraft  would  be 
mSi W*  lo"9  n«.  I hope  you  agree  that  such  research  has 

" iSirS  srtel?liS  Sfi“lie?*  ““ 

appreciate  their  position,  and  if  I were  in 
shoes,  I would  ^ve  had  the  same  reaction  from  an  unknown  person  making  unsupported 
J^ses  in  an  untimely  manner  to  the  solution  of  a politically  volatile  decision.  M a 

frustrated  bjr  the  fact  that  our  research  results  were  so 
****  ^ integrate  then  with  their  source  selection  efforts. 

US  i?Si.‘5's.“  *” - ““ 

maklliS  against  a system  I have  sworn  to  defend  without 

recommendations  for  its  improvement.  The  Air  Force  Academy  has  made  maey 
*5*  P“‘ se**™!  years.  The  only  ones  that  have  been 
IJfW  L *"*  i?®**  •“«*  * spoworw-usually  a general 

slln^I  always  translated  into  action  by  the 

**’•  research  and  the  implementation  phi  -es.  Exawles 
J*^**^*  Project  POL,  Project  EOQ,  Procurement  Quality.  RAO  Cost  Prediction, 
-5  ®“’^  opinion,  unsponsored  procurement  research  conducted  so  far  away  from  the  real 
IhIuia®!L?'?2ir*r®"^  |i«^provenTrustrating  and  non-rewarding.  Does  this  imply  that  we 
ImIim  ^ and  try  to  pose  solutions?  I sincerely  believe  that  we  must 

^ *>“J,‘>«Pore  we  expend  too  much  of  our  energy  and  enthusiasm, 

equally  enthusiastic.  This  is  extremely  difficult  for  us  at 
TOY  bud^  ^ acco^lish  as  we  have  a travel -limiting  teaching  schedule  and  very  small 

Horning  are  meant  to  be  constructive  in  nature,  for  we  at  the  Acadexy 
continue  our  research  efforts  in  the  procurement  area.  Being 
rhi<  *'*  "®^  ® position  to  strongly  market  our  efforts.  Many  of  you  in 

conduit  I-^M[I..4^fcf4®®"®*n!’  I®.****™  a way  the  unsponsor^  procurement  research  can  be 
H^Ihf^id  ,1.^*  5^^"^  *®  **®  are  proponents  of  the  established  system? 

HOW  ^ould  research  results  be  '■omnunicated  to  the  field?  How  can  we  Improve  the  attitude 

S*®®’*  selection  resSch 

*.*y***?  "**y  *’ave  overHding  political  considerations?  Must  we 

! * *<"“'^|y  hope  that  the  dialogue  we  have  today  will  help  all  of  us 
iccompilsn  what  we  so  dearly  want  to  do— constantly  Inprove  our  procurenent  systei. 
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EPILOGUE:  IvedUtely  after  these  comnents  t«re  made  at  the  sympolsua,  I received  tmo  very 
sincere  offers  to  allow  us  to  further  test  our  management  evaluation  model  during  two 
future  source  selections.  In  addition.  I made  contact  with  the  procurement  officer  of  the 
program  office  Involved  In  the  referenced  aircraft  source  selection.  He  invited  us  to 
provide  him  with  information  about  our  model,  and  I am  satisfied  that  our  research  will 
get  the  attention  It  should.  The  frustration  mentioned  during  this  talk  has  been  allevtaM 
because  of  the  participation  of  sincere,  educated  procurement  executives  who  truly  desire 
to  1^>rove  our  procurement  system.  I am  convinced  that  active  and  aggressive  mariwting  and 
cannicatlom  of  research  projects  Is  the  key  to  their  successful  Implementation.  For  us. 
ttls  6th  Anmnl  000  Procurement  Research  Symposlua  with  the  topic,  "Translating  Procurement 
Research  Into  Action*  has  been  a caag)1ete  success. 
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IMPROVING  THE  SOURCE  SELECTION  PROCESS 
BY  MEASURING  THE  HUMAN  RESPONSE  OF  PROPOSAL  EVALUATORS 
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Bob  Oycus 
Djrcus  Associates 
Tucson,  Arizona 
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ABSTRACT 


First,  data  is  presented  fron  the  Evaluator  Preference  Survey 
where  attitudes  and  evaluative  judgement  of  a qnasi*randoa  sample 
of  35  experienced  DoD  technical  proposal  evaluators  were  measured. 

This  data  indicated  that  government  personnel  have  very  favorable 
attitudes  toward  proposal  evaluations.  Strong  agreement  was  shown 
that  evaluations  are  interesting,  hard  work,  but  worthwhile  for  the 
government.  However,  other  Survey  data  indicated  considerable  room 
for  the  government  to  improve  proposal  evaluation  mechanics.  Most 
evaluators  indicated  they  reinterpreted  scored  evaluation  criteria. 
There  was  only  moderate  judgement  that  scored  evaluation  criteria 
and  rating  scales  were  "good”  and  "fair." 

It  is  suggested  that  the  government  improve  the  proposal  eval- 
Mtion  aspect  of  source  selection  by  conducting  experimental  research. 
The  experiments  would  consist  of  having  groups  of  experienced  evaluators 
score  experimental  and  real>world  proposal  material.  This  research 
would  improve  evaluation  rating  scales,  evaluation  criteria  for  scor- 
ingp  detenine  preferred  evaluation  Mechanics « and  ioprove  scoring 
discrimination. 

Feasibility  of  the  proposed  research  methods  is  demonstrated  in 
experiments  previously  conducted  by  the  author.  An  experiment  is  des- 
cribed where  the  same  experimental  proposal  material  was  scored  using 
seven  different  rating  scales.  Preliminary  data  is  presented  for 
scoring  differences  using  a truncated  versus  linear  adjective^numeric 
rating  scale.  It  was  found  that  the  linear  scale  gave  greatest  dis- 
crimination between  evaluation  scores--a  result  exactly  opposite  to 
commonly  held  government  belief.  Scores  were  compared  between  Navy 
used  relative  rank  scoring  and  Army  and  Air  Force  preferred  adjective- 
numeric  scoring.  Relative  rank  scoring  was  found  superior  in  forcing 
discrimination.  A proposal  scoring  difficulty  was  also  uncovered. 

Direct  readbacks  of  the  Statenent-of-Work  consistently  scored  near 
average  over  all  rating  scales  that  were  examined. 

End  product  of  the  proposed  experimental  research  would  be  a 
proposal  evaluation  guide  that  defines  a preferred  rating  scale,  and 
^^^®cts  the  evaluators  in  how  to  make  their  evaluation  scorings. 

Such  a guide  would  improve  the  quality  and  discrimination  of  proposal 
evaluation  scores, and  attest  to  the  practical  value  of  applied  pro- 
curement research. 


{ I INTRODUCTION 

Duriag  the  past  four  years  I have  been  conducting  eupirical 

V research  on  the  hunan  response  of  experienced  governaent  evalu- 
ators to  experiaental  proposal  aaterial.  The  aain  purpose  of  this 
research  has  been  to  iiqirove  contractor  proposals  submitted  to  the 
federal  governaent.  The  main  vehicle  for  doing  this  has  been 
. presentation  of  the  research  results  at  ay  public  seainars  "How 

I ) to  laprove  the  Evaluation  Score  of  Your  Technical  Proposals.** 


The  purpose  of  this  paper  is  to  show  how  the  sane  experiaental 


aethodology  can  be  used  by  the  governaent  to  iaprove  the  proposal 
evaluation  phase  of  the  source  selectiou  process.  Suggestions  for 
specific  research  will  be  aade,  and  feasibility  of  the  aethodology 
deaonstrated  by  presenting  soae  experiaental  results  froa  ay  pre- 
vious studies. 


c"  if 


GROUNDRULES  AND  DEFINITIOIS 


To  establish  a coaaoa  base  uf  understanding  some  definitions 
need  to  be  aade.  First,  we  will  be  concerned  .'^th  the  evaluation 
of  technical  proposals.  These  are  considered  to  contain  both  tech- 
nical and  aanagenent  discussions.  The  technical  proposals  are 
usually  rated  or  **scored**  by  evaluators  against  a set  of  established 
evaluation  factors.  Becnuse  of  the  diversity  in  the  use  of  such 
terns  as  evaluation  criteria,  evaluation  factors,  sub-factors,  etc., 
**evaluation  questions**  are  defined  to  be  the  actual  items  against 
which  the  proposals  are  scored.  **Evaluation  criteria**  are  defined 
to  be  the  stated  basis  on  which  the  proposals  are  scored.  These 
eppear  in  the  solicitation  and  are  usually  referred  to  there  as 
**evaluation  criteria**  or  *'award  factors.** 

Rating  aethods  have  been  examined  by  psychologists  in  some 
detail.  References  1 and  2.  There  are  four  basic  methods  of  rating: 
paired  comparison,  comparison  with  a set  of  examples  which  exemplify 
a range  of  the  attributes  being  considered,  rank  order,  and  rating 
(numerically,  by  adjectives,  or  colors)  on  soae  standard  scale. 

The  rating  methods  used  by  the  government  for  proposal  evaluation 
have  been  almost  exclusively  rank  order  and  rating  scale. 

The  government's  proposal  rating  scales  can  be  placed  into  the 
following  four  general  categories,  or  some  combination  of  them: 

NUMERIC  The  evaluator  using  this  rating  scale  merely  estimates  on 
a predetermined  numerical  scale  what  he  considers  to  be  an  appro- 
priate score  for  each  proposal  on  each  evaluation  question. 

ADJECTIVE-NUMERIC  This  rating  scale  ties  qualitative  adjectives 
to  numeric  rating  scale  scores.  This  helps  calibrate  the  sane 
numerical  scores  to  the  same  evaluative  judgement.  Often  the 
adjectives  contain  additional  amplifying  definitions. 
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im  rating  scales  are  often  truncated.  That 

linear,  but  instead  contains  a 

large  scoring  penalty  for  extremely  low  quality  proposals.  With 
truncated  scales  responses  that  are  marginally  acceptable  may , 

percent  of  the  total  scale  score  while 
Unacceptable  responses  score  nothing.  Many  government  people 
feel  that  truncation  produces  the  widest  separation  of  total 
evaluation  scores ••an  obvious  benefit  to  the  government. 


^CKOFF  Wi^this  rating  scale  the  evaluation  questions  receive 
only  one  of  two  possible  scores,  usually  "Acceptable”  or  "Un- 
acceptable."  This  rating  scale  is  often  used  in  evaluating 

a**b®ftted  in  Step  One  of  Two>Step  procurements 
where  it  is  only  necessary  to  determine  that  the  proposal  is 
technically  acceptable.  . . ^ 


AUECTIVE  On  this  rating  scale  a proposal  receives  one  of  a set 
of  adjective  ratings  on  each  evaluation  question.  Tne  adjectives 
may  also  ^clude  definitions  to  help  calibrate  scores.  Recently, 
color  codings  have  been  widely  used  in  place  of  adjectives.  Ad- 
J®ctives  or  colors  cannot  be  combined*  If  it  is  necessary  to 
combine  scores  then  numerical  equivalents  are  assigned,  and  the 
method  reverts  to  an  adjective-numeric  rating  scale. 


RELATIVE  RANK  is  also  used  to  rate  proposals.  In  relative  rank 
scoring  the  evaluators  rank  each  submitted  proposal  from  best  to 
poorest  response  on  each  evaluation  question.  Numerical  scores 
can  be  assigned  to  each  rank  position.  Relative  rank  is  heavily 
used  by  the  Navy,  but  not  much  outside  of  the  Navy. 


.^i®c^i^«->**umeric  rating  scales  are  predominantly  used  by 
the  Air  Force,  Army,  and  NASA.  The  following  rating  scale  usage 
was  obtained  from  an  analysis  of  about  55  proposal  evaluations 
conducted  by  these  agencies: 


RATING  SCALE 
Checkoff 
Adjective 
Adjective-Numeric 
Numeric 


PERCENT  OCCURRENCE 
6 

12 

59 

23 


The  sa^le  contained  about  equal  representation  from  the 
three  agencies  with  fairly  random  distribution  geographically 
and  among  technical  disciplines.  The  sa>q>le  is  too  small  to 
accurately  reflect  the  real  occurrence  rates,  but  is  probably 
sufficient  to  show  that  adjective-numeric  rating  scales  are 
predominently  used  in  proposal  evaluations  up  to  a few  million 
dollars  by  these  agencies. 
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EVALUATOR  ATTITUDES  CONCERNING  .mi  EVALUATION  PROCESS 


I recently  completed  an  Evaluator  Preference  Survey  (Reference 
3)  which  measured  attitudes  and  evaluative  judgement  of  a quasi- 
random  sample  of  33  experienced  DoD  proposal  evaluators.  Included 
in  the.  Survey  questionnaire  were  two  items  concerning  the  evaluation 
process.  These  two  items  and  the  evaluators  scorings  are  shown  in 
Figure  1. 

The  evaluators  evidenced  very  favorable  attitudes  toward  pro- 
posal evaluations.  The  majority  of  the  evaluators  appeared  to 
find  proposal  evaluations  interesting  and  not  boring.  There  was 
strong  agreement  that  proposal  evaluations  are  hard  work,  but  are 
worthwhile  for  the  government.  These  favorable  attitudes  might 
imply,  or  are  in  keeping  with,  diligence  in  evaluating  proposals. 

It  should  be  added  that  checks  included  in  the  questionnaire  in- 
dicated that  honest  rather  than  "motherhood**  responses  were  ob- 
tained in  the  survey. 

. * . ► 

Seven  questionnaire  items  were  included  to  probe  evaluator 
pre-selection  bias.  The  scores  implied  that  7S  to  80  percent  of 
the  evaluators  appeared  to  consider  the  submitted  proposal  more 
important  than  prior  knowledge  of  the  submitter.  Consequently, 
one  out  of  four  or  five  evaluators  evidenced  strong  pre-selection 
bias.  i.e..  prior  knowledge  of  a submitter  was  more  iaportant 
than  the  sul^tted  proposal. 


EVALUATOR  ATTITUDES  CONCERNING  PROPOSAL  EVALUATION  MECHANICS 

Two  items  were  included  in  the  Survey  questionnaire  to 
measure  attitudes  toward  the  mechanics  of  the  evaluation  pro- 
cess. Scorings  of  these  items  are  shown  in  Figure  2. 

Apparently,  there  is  considerable  room  for  the  government 
to  improve  its  evaluation  processes.  Most  evaluators  indicated 
that  they  made  some  reinterpretation  of  the  evaluation  criteria 
(i.e..  the  things  the  evaluators  actually  scored--the  evaluation 
questions).  There  was  only  a moderate  judgement  that  the  '*evalu- 
ation  questions"  and  rating  scales  were  "good"  and  "fair." 


SUGGESTED  RESEARCH 

A potentially  powerful  research  tool  is  available  to  improve 
the  qualif*  of  proposal  evaluations.  This  tool  is  application  of 
the  methOL  • /y  of  experimental  psychology  to  proposal  communications 
In  essence,  experimental  and  real-world  proposal  material  would  be 
scored  by  groups  of  experienced  evaluators.  By  measuring  the  evalu- 
ators' responses  the  government  could  examine  many  things,  for 
example: 

1.  By  scoring  the  sane  proposal  material  with  different 
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rating  scales  the  government  could  determine  preferred  ratine 
scales.  Rating  scales  would  be  selected  based  on  such  things 
as  greatest  separation  of  total  evaluation  scores  (discrimi- 
nation) and  inter-rater  score  dispersion  (reliability), 

2.  Evaluation  questions  could  be  improved  by  scoring  the  sane 
proposal  material  against  alternate  forms  of  an  evaluation 
question.  Preferred  evaluation  questions  could  be  selected 

.in  terns  of  utility  for  discrimination  and  smallest  dispersion. 

3.  Proposal  material  that  presents  scoring  difficulties  could 
®®t®cted.  Scoring  guidelines  could  then  be  fomulated  to 

aid  evaluators  in  scoring  these  problem  areas. 

4.  Evaluation  procedures  could  be  improved  by  scoring  the 

sane  proposal  material  under  different  procedural  method- 
ologies. For  example,  proposals  could  be  independently 
scored  with  subsequent  discussions  and  scoring  changes 
allowed  versus  having  the  evaluators  discuss  the  proposals 
together  in  a group . r 


This  is  only  a partial  list  of  the  many  questions  that  could 
attacked.  This  suggested  research  appears  very 
cost  effective.  That  is,  the  experimentation  would  be  relatively 
inexpensive,  but  could  pay  big  dividends  in  improving  the  quality 
of  proposal  evaluation  for  source  selection.  ^ ^ 


The  intrat  of  congress  promulgated  through  laws  and  regulations 
is  source  selection  based  on  the  proposal  Mst  advantageous  to  the 
f overrent • If  the  desire  is  honest  and  objective  source  selection 
toen  improv^  evaluation  of  proposals  can  be  of  great  help.  If  the 
fs  mininiration  of  such  things  as  contract  award  protests, 
then  the  proposed  research  nay  appear  unnecessary.  It  should  be 
j minimization  of  award  protests  has  not  been 

established  by  congress  as  a procurement  objective.  Also,  recent 
non-numerical  scoring  of  proposals  (e.g.,  color  coding)  has  greatly 
*®treased  contractor  suspicions  of  dishonesty.  Cynicism  on  the 
part  of  the  contracting  community  can  have  strong  adverse  effects 
for  the  government  that  far  outweigh  defending  a few  protests. 


EXPERIMENT  DESCRIPTION 

I have  conducted  some  preliminary  experimental  studies  which 
demonstrate  the  feasibility  of  the  proposed  research.  The  subjects 
in  the  experiments  were  participants  at  ny  public  seminars.  During 
the  seminars  the  subjects  were  provided  a typed  four  page  Case 
Study.  The  Case  Study  asked  the  subjects  to  assume  that  they  were 
goverMent  employees  assigned  to  a technical  evaluation  committee 
on  a 1200,000  test  program  procurement.  The  subjects  were  then 
asked  to  score  five  hypothetical  contractor  proposal  responses 
against  the  following  evaluation  question:  "Has  the  contractor 
provided  adequate  photographic  support  for  the  test  program?" 
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The  five  proposal  discussionii  responded  to  the  following  State* 
■ent-of-Kork  item:  “During  conduct  of  the  tests,  the  contractor 
shall  provide  high  speed  motion  picture  coverage  of  the  tests 
(black  and  white,  16mm  film  at  200  frames  per  second).  These 
films  will  be  provided  to  the  cognizant  DCAS  technical  representa- 
tive within  two  weeks  following  each  test.“ 

The  five  proposal  responses  to  this  Statement*of*1fo;ic  item 
were  structured  to  represent  typical  approaches  often  used  in 
proposals.  They  may  be  summarized  as  follows: 

Contractor  A*s  response  was  a direct  readback  of  the  Statement* 
of -Work.  .L 

Contractor  B proposed  to  paint  the  test  item  according  to  a multi- 
color  code;  and  photograph  the  tests  with  color  film,  “to  illustrate 
the  actual  test  effects."  The  type  of  film  was  specified.  Com- 
pliance was  indicated  to  filming  at  200  frames  per  second,  and 
delivering  the  film  within  two  weeks  to  the  DCAS  representative. 

t) 

It  was  originally  intended  that  this  response  would  represent 
a “gold-plating"  approach.  That  is,  the  requirement  was  for  black 
and  white,  but  the  contractor  was  going  to  provide  something  above 
the  minimum  requirement,  namely,  color  film  at  an  implied  increase 
in  cost.  However,  a second  legitimate  interpretation  of  this  pro- 
posal response  was  also  encooBtered.  The  color  film  may  not  be 
technically  acceptable.  For  example,  color  film  at  200  frames  per 
second  does  not  provide  the  same  resolution  as  black  and  white 
film.  Consequently,  color  film  night  not  be  technically  acceptable 
if  resolution  is  i^ortant. 

Contractor  C*s  proposal  was  intended  to  be  a good  response.  The 
contractor  indicated  how  he  would  perform  the  work  by  indicating 
camera  mounting  and  film  processing  arrangements.  In  addition, 
he  gave  the  government  a cost-free  3-day  tum-around  time  on  film 
delivered  to  the  DCAS  representative,  and  concurred  that  the  photo- 
graphic coverage  was  a necessary  requirement  based  on  his  previous 
testing  experience. 

Contractor  D used  a give-us-the-money-and-we-will-do-whatever-you- 
want  approach.  He  proposed  to,  “provide  high  speed  notion  picture 
coverage  of  each  test,  and  provide  the  film  to  the  DCAS  represent- 
ative within  the  required  two  week  period,"  and  to,  "fully  coop- 
erate with  the  government  to  fill  any  photographic  needs  that  may 
arise." 

Contractor  E*s  response  was  considered  to  be  a good  "how"  response. 
The  cohstractor  indicated  how  the  cameras  would  be  mounted  and  the 
film  processed. 
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VARIATION  OF  RATING  SCALES 

The  five  proposal  approaches  were  scored  by  different  groups 
of  subjects  using  different  rating  scales.  Although  extraneous 
variables  were  not  strongly  controlled  in  these  experiaents  the 
data  was  sufficient  to  serve  the  purpose  of  a preliainary  study. 
The  following  seven  rating  scales  were  exaained: 

Group  1 used  the  truncated  adjective-nuaeric  rating  scale  shown  in 
Figure  3.  This  rating  scale  was  an  adaptation  of  one  recoaaended 
in  Reference  4 for  use  by  the  Aray. 

Croup  2 used  the  saae  adjective-nuaeric  rating  scale  as  Groiqi  1, 
only  the  nuaerical  scores  were  changed  to  a linear  scale.  Under 
this  change  Excellent  was  100.  Very  Good  was  83.  Good  was  67. 
Average  was  50.  Fair  was  33.  Poor  was  17.  and  Unacceptable  was 
zero. 

Group  3 scored  the  proposal  responses  by  relative  ranking  the 
subaissions  froa  1 (best)  to  S (worst). 

Group  4 used  the  experiaental  rating  scale  shown  in  Figure  4. 

Croup  5 used  a set  of  seaantic  differential  scales  I wanted  to 
test.  The  seaantic  differential  was  developed  by  Osgood.  Refer- 
ence S.  and  is  widely  used  by  psychologists  in  attitude  and 
other  studies.  It  appears  adaptable  to  proposals,  but  has  not 
been  used.  The  subjects  using  this  rating  scale  were  asked 
to  rate  the  "Photographic  Support  of  Test  Prograa"  on  a 7 point 
contlnuua  between  opposite  adjective  poles.  The  opposite  poles 
used  were:  inadequate-adequate,  bad-good,  and  hazy-clear. 

Group  6 scored  the  proposal  subnissions  on  a seaantic  differential 
scale  using  only  the  poles  "adequate"  and  "inadequate." 


Group  7 scored  the  five  proposal  submissions  on  a semantic  differ- 
ential  scale  with  poles  of  "good"  and  "bad." 

To  date,  data  has  been  analyzed  partially  only  for  the  first 
three  groiq>s.  However,  some  interesting  results  have  emerged  froa 
this  limited  preliminary  data  analysis. 

First,  the  truncated  rating  scale  compressed  the  evaluation 
scores  relative  to  the  linear  rating  scale--a  result  contradicting 
the  reason  given  for  using  truncation.  This  can  be  seen  froa  the 
mean  scores  of  Groups  1 and  Z‘  shown  in  Figure  S.  The  range  from 
highest  to  lowest  score  using  the  truncated  rating  scale  was  54.7. 
This  coapares  to  a range  of  64.3  using  the  linear  scale. 

Second*  the  linear  scale  was  superior  to  the  truncated  scale 
in  other  aspects  of  discrimination. 

A.  The  linear  scale  was  superior  in  separating  the  scores 
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of  the  "good”  proposal  responses,  C and  E,  froa  the  two 
"poor-*  responses,  B and  D.  The  separation  between  responses 
E and  B,  the  weakest  of  the  "good"  and  the  strongest  of  the  > 
"poor"  was  46.1  using  the  truncated  scale.  However,  on  the 
linear  scale  this  separation  increased  to  S3. 6.  < 

T I I I 

B.  Better  discriaination  was  obtained  between  the  two  best  re* 
sponses,  C and  E,  when  the  linear  scale  was  used.  However, 
this  last  discriaination  nay  not  really  be  significant. 
Duncan's  Range  Tests  were  run  between  the  aean  scores  of 
each  group.  The  tests  showed  that  the  differences  between 
all  aean  scores  of  Groi^  1 were  statistically  significant 
at  the  five  percent  probability  level  except  between  C and 
E.  In  Group  2 the  differences  between  all  aean  scores  were 
statistically  significant  except  between  B and  D,  and  again 
between  C and  E. 


In  this  experiaent  the  govemaent's  objectives  of  greater  separa* 
tion  of  evaluation  scores,  better  discriaination  between  proposals  in  j 
i the  coapetitive  zone  versus  those  not  in  the  coapetitive  zone,  and 

' — ■ better  discriaination  aaong  proposals  in  the  coapetitive  zone  were 

better  achieved  with  the  linear  rather  than  the  truncated  rating  scale. 

UMore  study  of  rating  scale  truncation  appears  warranted.-  Recoaaended 
additional  study  would  include  scoring  real-world  proposals  by  evalu- 
ators using  linear  versus  truncated  rating  scales. 

. _ <3 

COWARISON  OF  AOJECTIVE-NDNERIC  TO  RELATIVE  RANK  SCORING 

Relative  rank  scoring  was  conpared  to  adjective -nuneric  scoring. 

( To  do  this,  the  adjective-nmeric  scores  of  Groiq>s  1 and  2 were  con- 

verted  to  relative  rank  scores  based  on  the  decending  order  of  each  . 

subject's  nuuerical  scores.  Relative  rankings  were  not  deterair.ed 
in  cases  where  subjects  scored  two  or  sore  responses  nunerically  the 
I,  ; sane.  After  deletion  of  these  cases,  28  relative  rank  scorings  were 
derived  for  Group  1,  and  26  relative  rank  scorings  were  derived  fo« 
Group  2. 

Analysis  of  the  rank  data  showed  no  statistically  significant 
differences  between  the  rankings  of  Groups  1 and  2 versus  Group  3. 

Tlds  analysis  consisted  of  conducting  Duncan's  Range  Tests  between 
, the  neans  of  the  three  different  groups  on  each  proposal  response. 

' There  were  no  differences  that  were  significant  at  the  five  percent 

probability  level. 

I One  potentially  inportant  difference  between  relative  rank  and 

adjective-nuneric  scoring  was  observed.  A large  percentage  of  the 
subjects  rated  two  or  nore  of  the  responses  the  sane  when  they  em- 
ployed adjective-nuneric  scoring.  Abo.;,  half  the  subjects  in  Group 
1*  27  out  of  SS,  scored  two  or  nore  responses  the  sane.  In  Group 
2,  11  out  of  37  subjects,  30  percent,  scored  two  or  nore  of  the  I 

responses  the  same.  This  conpares  to  only  3 of  the  54  subjects,  6 
percent,  in  Group  3 that  were  unable  to  relative  rank  the  proposal 
^ responses  differently.  Consequently,  relative  rank  scoring  nay  be 

superior  to  adjective-nuneric  scoring  in  discrimination. 


n 


ml 


am.  5“"’'  scoring  has  one  inherent  weakness.  The  evalu^ 

make  relative  evaluative  judgements.  With 
relative  judgements  one  proposal  submission  will  alwavs  rank 
first  regardless  of  how  ••poor"  all  the  subSirsions  ^ S5  one 

regardless  of  how  ••good^^  *11  the  prooosals 
ml  if*  i*  fenf^al.  ranking  provides  the  least  information  because 
one  can  obtain  identical  sets  of  rankings  for  proposals  which  5m 
quite  similar  and  proposals  that  are  extremely  different. 

The  best  approach  ulght  be  a rating  scale  that  asks  the 

consciously  make  both  absolute  and  relative\v*Iu- 

Sli5?«J^J**“*5***  couinj  e»eJiIi5tally 

o5  its^^Me.*”**  require  giving  directions  to  the  evSluators 


DETECTION  OF  SCO.’UNG  DIFFICULTIES 

The  preliminary  experimental  work  has  surfaced  an  examole  of  . 
a proposal  scoring  difficulty.  My  personal  feeling  is  tha^read- 
Statenent»of-lfork  aie  very  poor  resoonses  and  chonid 

n5‘stat^5t  v**w***i.i^5*-  inconsequential  requirements) 

The  Stateaent*of*1fork  should  describe  •Sdiat**  is  to  be  done  The  ^ 

should  tell  the  government  ••how*’  the  program  reoulrements 
will  be  met.  Statement>of-Nork  readbacks  do  not  describe  ••how"  th* 
requirements  will  be  met  and  should,  in  my  opinloS!  5roro  roJJ  iJll! 

It  was  my  observation  that  readbacks  on  the  Statemcnt-of-Nork 
usually  scored  about  average  in  evaluations  that  I participated  in 
as  a government  employee.  The  experimental  work  has  confirmed  thi« 

°!!  of  direct  riad555ks  5f “he  inJewi? 

presented  in  Figure  6.  The  Mock  Proposal  data  points 
llLt  t “ 1 had  29  experienced  government  e!alu- 

*S®ru.®®***  competing  nock  proposals.  Additional  descriptions 
?in5l5f*.****!*’^"*"**  References  6 and  7.  The 

of?w5rt"^o»5J«^  *^1**°^%^*  direct  readbacks  of  the  Statement- 

«.  Shs:; 


END  PRODUCT  APPLICATION  OF  THE  PROPOSED  RESEARCH 

the  useful  application  of  the  suggested 
research  would  be  improved  evaluation  questions  and  a short  evalu- 
ation guide.  The  evaluation  guide  would  instruct  evaluators  on  how 
i5d*f5r*  Pr®P®*«l  "aterial.  The  rating  scale  would  be”r55en?5d, 
and  its  operation  described.  Applicable  procurement  regulations 
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Directions  would  be  given  for  scoring  difficult  material.  For  exaap 
readbacks  of  the  Statement-of-Work  on  substantive  material  might 
receive  a uniform  score  of  2S. 

( 

Success  of  the  proposed  evaluation  guide  is  almost  guaranteed. 
That  is,  it  is  well  known.  Reference  1,  that  training  (which  the 
guide  would  provide)  increases  both  intrarater  and  interrater  scorii 
reliability.  The  guide,  resting  on  foundations  of  empirical  pro* 
curement  research,  shesld  greatly  improve  the  government's  source 
selection  process,  and  attest  to  the  practical  value  of  procurement 
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THE  EVALUATION  PROCESS,  Survey  Itcns  No.  14  and  37 

OBJECTIVE:  Examine  evaluator  attitudes  toward  the  evaluation  proeessm 

QUESTIONNAIRE  ITEM  NO.  14: 


Proposal  evaluations  are  a bore. 

disagree  ; : : : : agree 

RESULTS:  Vv  i - 


. - * - ; - , r*  4.  7^<T 

* * • - * i A \ > -* 

or  unable  to  score 
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*•  •'  f. : disagree 
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n not  scored 
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qUESTlONKAlRE  ITEM  NO.  57:  ' ~ ' • i o .. 

Proposal  evaluations  are  hard  aork,  but  are  worthwhile  for  the 
govemaent.  , 

I disagree  : : : : s t agree  or  unable  to  score 


RESULTS: 


1'/ 
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not  scored 
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FIGURE  1.  Result. s Fron  the  F.valuntor  Preference  Survey  Showing 
Favorable  Attitudes  Toward  the  Proposal  Evaluation 
Process 
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TllE  EVALUATION  PROCESS,  Survey  I teas  No.  22  and  43 


OBJECTIVE: ^ Examine  evaluator  attitudee  toward  the  meohaniee  of  the 
evaluatxon  proeeee.  * 

QUESTIONNAIRE  ITEM  NO.  22:  ; • ‘ |^  . r. 


\>’l 


i- 


RESULTS: 


: often 


i--;  5:^ 


never 


or  ^ unable  to  score 


704466  1 


often 


□ 


not  scored 


QUESTIONNAIRE  ITEM  NO.  43: 


»at  is  your  opinion  of  the  evaluation  criteria  and  ratine  scales  used 
in  proposal  evaluations?  ^ 


subjective  i 
food  : 


: : unfair 


objective 

bad 


or  ^ unable  to  score 
or  __  unable  to  score 
or  unable  to  score 


RESULTS: 


subjective 


□ 


Jib 

LL.i  n obj 


>583642 


objective 


CD  not  scored 
2 


bad 


cd 


1 2 6 5 0 7 1 


eood 
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not  scored 


FIGURE  2.  F.valuator  Preference  Survey  Data  Indicatinf!  A Need 
For  Inprovenent  in  Proposal  Evaluation  Mechanics 
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100  Excellent  (a  very  conprehensive,  co^>lete  response  to  every 
detail  of  a respective  area) . 

90  Very  Good  (a  proposal  with  clear  responses  to  all  requirements 
ot  a respective  area. 

80  Good  (a  proposal  with  definable  detail  in  excess  of  the 
minimum  requirements). 

70  Average  (minimum  acceptability,  bidder  meets  minimum 

^ requirements) . 

60  Fair  (lack  of  clarity*inplications  or  vague  indication  that 
capability  is  present  in  accordance  with  RFP) . 

SO  Poor  (incomplete-omission  of  minor  details-omissions  or 
^ misunderstandings  of  requirements  in  minor  areas). 

00  Unacceptable  (gross  omissions  or  failure  to  respond  to 
requirements  of  major  areas).  ^ 

FIGURE  3.  The  Trucated  Adjective-Numeric  Rating  Scale  Used  by 
Subjects  in  Groiqt  1 


'vr.. 


/ 


. .A  i'  . ■*. 


T 


T 


'l- 


i 


^ * ' 


1 


♦30  Excellent 


♦20  Very  Good 


Will  Beet  Binimun  requirements  with  an  almost 
assured  probability  of  sucess;  and/or,  offers 
substantial  advantage(s)  over  Binimum  require- 
ments at  no  additional  or  at  a reduced  cost* 

High  probability  that  Binimua  requirements  will 
be  met;  and/or,  offers  some  advantage(s)  over 
minimum  requirements  at  no  additional  or  at  a 
reduced  cost. 


10  Good  Good  probability  that  minimum  requirements  will 

be  met;  ud/or,  offers  small- potential  advantage(s) 
over  minimum  requirements  at  not  additional  cost. 

00  Average  Should  meet  minimum  requirements.  * 


•10  Fair 


! 


-20  Poor 


Some  question  that  it  will  Beet  BiniBUB  require-  . 
, _ ments;  and/or,  exceeds  Binimum  requirements  at 

some  additional  cost  to  the  government;  and/or 
.1 . , . proposal  is  ambiguous  or  partially  unclear. 

High  probability  that  minimum  requirements  will 
not  be  met;  and/or,  exceeds  minimum  requirements 
at  an  additional  cost  to  the  government;  and/or. 
proposal  ,is  poorly  presented  (unclear  or  larce 
omissions) . 


•30  Unacceptable  Almost  assuredly  will  not  meet  minimum  require- 
ments; and/or  exceeds  minimum  requirements  at  a 
significant  additional  cost  to  the  government; 
and/or,  fails  to  address  requireBent(s) . 

FIGURE  4.  The  Experimental  Rating  Scale  Used  by  the  Subjects  in 
Group  4 
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FIGURE  S.  Scores  for  the  Sane  Five  Proposal  Responses  Using  a 
Truncated  vs.  Linear  Adjective -Numeric  Rating  Scale  t.. 
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FIGURE  6.  Scoring  of  Proposal  Material  Shoving  Mean  Scores  of  Direct  i 

Readbacks  of  the  Stateaent-of-Work  Consistently  Near  the  Rating 
Scale  Definition  of  "Average" 
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EXPfltlrtPi*  of  A r.ig»  Sm«^v  nr^  thm  n«^  n* 

■Eerlamifl!!.  * Tnri>r.rivr.^  -a  T«><?hnf>ln«iea1 

Charles  Hulick 

Chief,  Procureaent  Area 

C*per1aiental  Technology  Incentives  Prooraa 

National  Bureau  of  Standards 

•V  •-  . 1 

Introduction  ' 

purpose  of  this  paper  is  to  document  a procurement  experiment 
jointly  sponsored  by  the  Experimental  Technology  Incentives  Pro- 
gram and  tno  Federal  Supply  Service,  wherein  performance  incen- 
tives were  employed  in  an  attempt  to  achieve  technological  innova- 
tioiie  The  paper  will  first  briefly  describe  the  Experimental 
Technology  Inccnti/es  Progranip  in  particular  the  procurement 
.ortion^  ars ! then  document  the  experiment  design  and  implement:;- 
tion  fhases.  Pinallyp  issues  to  bo  resolved  in  the  evaluation 
f*hase  will  be  highlighted. 

11^  Experimental  It^chnology  Incentives  Program 

In  hi3  March  1972  Science  and  Technology  and  1973  Budget  Messages, 
whe  President  called  for  a terting  of  possible  partnership  arrange- 
ment j among  various  Government  levels,  private  firms  and  univer- 
sities, and  the  initiation  of  a scries  of  experiments  to  find 
better  ways  of  stimulating  private  investment  in  research  and 
development.  This  was  the  birth  of  the  Experimental  Technology 
Incentives  Program  icno%m  as  ETIP. 

This  program,  which  is  under  the  \ational  Bureau  of  Standards, 
is  part  of  a continuing  effort,  oi:  the  part  of  Federal  Government, 
to  find  ways  in  which  it  can  %rork  as  a more  effective  partner  with 
the  private  sector  of  our  society  in  the  development,  application, 
and  transfer  of  science  and  technology  to  strengthen  the  Nation *s 
economy  and  improve  the  quality  of  life. 

The  ETIP  objective  is  to  conduct  an  informed  inquiry  into  the 
relationship  between  Governmental  actions  and  technological 
innovation  in  the  private  sector.  The  purpose  of  the  inquiry 
is  to  discover  and  test  appropriate  Governmental  policies  and 
practices  which  could  stimulate  desirable  innovation  in  the 
civilian  economy  and  thus  contribute  to  the  solution  of  national 
problems.  The  general  method  of  conducting  this  inquiry  is  to 
work  in  close  coopv^ration  with  appropriate  Government  agencies 
in  the  identification,  analysis,  testing,  and  evaluation  6f 
V'otential  policy  related  incentives  for  innovation.  The  parti- 
cular policies  addressed  will  be  ot  significant  interest  to  the 
cooperating  agency  and  will  represent  a general  process  which 
has  application  in  other  areas. 
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This  experiniental  program  is  desig..ed  to  gain  insight  into 
practical  problems  by  developing  unique  information  from  actual 
experience  to  be  used  as  the  basis  for  policy  recommendations. 

The  Program  is  divided  into  three  major  policy  areas:  Regulation, 
Economic  Assistance,  and  Procurement.  * . 

Procurement  Area  — t 

In  the  Procurement  Area,  we  are  testing  the  hypothesis  that  the 
use  of  procurement  incentives  in  government  contracting  can 
encourage  suppliers  to  incorporace  new  technology  in  their 
products  being  offered  both  to  the  government  and  the  c >arical 
sector.  It  is  felt  that  current  procurement  methods  create 
to  this  technological  improvement,  and  the  use  of 
incentives  can  remove  these  barriers.  As  an  example,  the  tradi- 
tional low  bid  approach  used  in  contracting  provides  no  incentive 
? supplier  to  do  better  than  the  minimum  prrfcr7»aiicc  level 
jpecified,  thus  ensuring  that  the  government  will  always  be  in 
the  position  of  receiving  minimum  quality  level  products.  The 
use  of  life  cycle  costing  can  change  this  situation  in  that  suppliers 
are  put  on  notice  that  the  government  is  interested  in  thxf  best 
or  optimum  level  of  performance  rather  than  the  minimum;  in  the 
case  of  procuring  appliances,  this  means  that  the  more  energy 
efficient  appliances  will  fare  better  in  the  bid  evaluation  pro- 
cess since  they  will  have  lower  overall  operating  coats.  Thus, 
the  low  bid  barrier  has  iieen  removed,  and  suppliers  are  rewarded 
for  better  quality  products. 

Before  undertaking  a specific  procurement  experiment  with  an 
agency,  several  factors  must  be  considered: 

o The  element  of  technology  must  te  identified,  whether  it  be 
energy  conservation,  higher  ocrforraance,  longer  life  or  what- 
ever. It  should  be  clear  at  the  outset  exactly  what  is  to 
be  improved  through  the  experiment. 

o The  technology  element  must  be  ranked  and  compared  in  both 
the  Government  and  commercial  markets  to  identify  if  the 
Government  leads,  lags  behind  or  is  even  with  the  commercial 
market  version  of  a product  or  product  class. 

o Agency  goals  with  respect  to  the  desired  technology  element 
need  to  bo  identified  and  established  as  part  of  the  experi- 
ment. The  ETIP  goal  of  private  sector  technological  innova- 
tion as  the  ultimate  goal  of  the  experiment  is  a long-range 
one;  since  the  needs  of  participating  agencies  often  require 
a more  immediate  payoff,  products  tend  to  be  selected  for 
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experiments  where  the  Government  version  of  the  product  lags 
behind  the  commercial  market  version.  The  result  is  two  sets  | 
of  goals  being  established#  short  term  and  long  term.  In  the 
short  term#  the  agency  goal  is  set  at  bringing  the  technology 
level  of  their  product  up  to  that  of  the  commercial  market. 

Once  this  has  been  achieved#  the  long  term  ETIP  goal  of  pri- 
vate sector  innovation  through  procurement  incentives  then  . it/. 
becomes  appropriate.  . - 4:  * 


. • -4  _ i • • 


o Policy#  procedural#  traditional  and  other  barriers  to  agency  goal 


and  technology  achievement  must  be  identified.  Such  barriers 
include  risk  of  buying  a product  the  customer  agencies  won*t 
use,  additional  funds  to  cover  the  extra  cost  of  a new  product 
submitted  in  response  to  an  incentive#  and  lack  of  properly 
trained  personnel. 


o lETIP  resources#  under  the  experimental  fraraevrork  can  be 


directed  toward  developing  an  experiment  to  achieve  goals 
and  objectives  while  reducing  barriers  to  innovation  in  the 


procurement  process.  i . ' <4- 4 .c 

, - i . • > * * •- 

•*  _ * 

Once  these  factors  have  been  considered  the  design  of  the  procure- 


ment experiment  begins.  Here  the  ETIP  procurement  program  staff 
works  with  the  participating  agency  staff  to  obtain  agency  com- 
mitment for  the  duration  of  the  experiment#  design  the  details 


Ui  of  the  procurement  incentive  to  be  tested,  if  applicable,  inter- 

act with  industry  and  plan  the  evaluation  of  the  experiment  after  . 


each  procurement. 


i • 


I j Lawnroower  Project 

bawnmowers  were  selected  for  a procurement  experiment  because 


they  were  an  item  that  the  government  purchases  in  large 
quantities#  an  item  that  has  an  excellent  commercial  transfer- 
ability factor#  and  one  that  had  (at  the  time)  national  attention 
regarding  the  need  for  improved  safety  and  lower  noise  levels, 
rhe  ETIP  st.^ff#  therefore#  developed  in  coordination  with  Federal 
Supply  Service  a procurement  project  which  would  attempt  to  en- 
courage suppliers  to  offer  innovative  ideas  in  the  areas  of 
safety  and  noise  control.  The  design  of  the  project  called  for 
a definite  quantity  purchase  of  10,000  21*  rotary  lawi^wers# 
which  in  itself  was  thought  to  be  a sufficient  incentive  to 
industry  since  normal  contracting  procedures  involved  indefinite 
term  contracts.  The  ETIP  project  plan  for  this  experiment  was 
formally  approved  in  October  1973  and  Federal  Supply  Service 
issued  their  bid  document  to  industry  on  Hay  29#  1974. 
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In  developing  the  statement  of  requirements  for  the  project, 

FSS  worked  closely  with  representatives  of  the  Consumer  Product 
Safety  Commission  (CPSC)  and  the  Environmental  Protection  Agency 
(EPA).  The  purpose  here  was  to  develop  goals  in  the  area  of 
safety  and  noise  which  would  be  both  realistically  obtainable 
and  reflect  an  improvement  over  existing  technology.  In  the 
area  of  safety,  the  principal  requirements  was  for  a device  to 
be  installed  on  the  mower  which  would  automatically  cause  the 
blade  to  stop  within  3 seconds  after  the  operator  left  his  station. 
This  requirement,  strongly  endorsed  by  CPSC,  became  known  as 
the  "dcadiiian's  clutch."  In  the  area  of  noise,  it -was  felt  that 
based  on  available  conmiercial  mowers  ("Lavmmowers:  Noise  and 
^ost  of  Abatement,"  EPA,  June  1974)  that  a reasonable  target 
goal  should  be  85dBA  (bids  above  that  level  would  be  rejected) . 

It  was  decided  that  the  two  step  formal  advertising  procedure  i 

%rould  best  meet  the  needs  of  this  situation  in  that  the  government 
was  calling  for  technology  which  was  not  currently  available; 
therefore,  step  one  of  the  procurement,  the  call  for  technical 
proposals  issued  March  19,  1974^  with  a closing  date  of  May  29, 

1974.  Eight  proposals  were  received;  however,  the  technical  evalua- 
tion ’of  the  proposals  revealed  that  none  of  them  were  able  to  meet 
bo^  the  safety  and  noise  requirements  of  the  project.  The 
principal  factor  which  caused  the  inadequate  response  was  the  ^ 

lack  of  sufficient  lead  tinie  for  suppliers  to  do  any  significant 
developmental  work.  To  overcome  this  problem,  it  was  agreed  by 
project  participants  to  issue  a "letter  of  intent"  to  industry 
which  would  substantially  increase  the  leadtime  between  "bid  * . ^ 

issue  date*  and  the  closing  date  for  technical  proposals. 

Accordingly,  FSS  issued  in  September  1975  a letter  of  intent 
to  industry  which  announced  the  future  purchase  of  10,000  mowers 
and  call  for  technical  proposals  to  be  submitted  by  January  1976. 
Additionally,  a sliding  point  scale  was  incorporated  into  the 
method  of  award  which  on  a scale  of  zero  to  10  rewarded  bids 
which  offered  noise  levels  at  less  than  88  dBA.  The  procedure 
called  for  adding  to  the  suppliers  purchase  price  a maximum 
of  10  points  at  the  98  dBA  level  down  to  adding  zero  at  the 
80  dBA  level,  with  award  to  be  made  to  the  lowest  overall 
combination  of  purchase  price  and  noise  weight  factor.  The 
"deadman's  clutch"  requirement  remained  constant. 


Response  to  this  statement  of  requirements  were  few  in  number 
and  while  they  appeared  to  meet  the  noise  level  criteria,  they 
did  not  meet  the  clutch  requirement.  After  consultations  with 
membi^rs  of  CPSC  wherein  it  was  learned  that  an  industry  standard 
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requiring  the  clutch  device  was  at  least  a year  away,  it  was 
decided  that  the  clutch  requirements  should  be  dropped  and  con* 
centrate  solely  on  noise.  Additionally,  86  dBA  was  set  as  the 
new  maximum  level.  Suppliers  were  advised  of  this  fact,  and  a 
new  submissions  date  of  March  30,  1976,  was  established. 

Eight  proposals  were  received  under  the  revised  requirements  and 
technical  evluation  begun.  Proposals  reviewed  were  both  adequate 
and  encouraging  from  a noise  level  viewpoint,  and  price  proposals 
were  requested  in  June,  1976.  Final  award  was  made  to  AMF 
Industries  in  February  1977  for  a 21"  rotary  mower  which  was 
rated  (by  AMP)  at  82  dBA.  Terras  of  the  contract  called  for 
validation  of  this  noise  level  at  the  pre-production  inspection 
date  of  May  23,  1977  and  for  delivery  to  start  in  July,  1977. 

It  is  interesting  to  note  that  award  was  not  made  to  the  lowest 
bidder  in  terms  of  initial  purchase  price,  which  was  $92.50 
with  a sound  level  of  86.  Instead,  airard  was  made  for  the 
model  priced  at  $102.45,  but  with  a sound  level  of  82.  Thus, 
the  sliding  scale  of  point  values  was  in  fact  utilized  in  the 
contracting  process.  * 

What  remains  to  be  done,  from  the  ETIP  prospective,  is  to 
evaluate  this  procurement  from  both  the  agency  and  comiLercial 
impact  points  of  view.  What  impe^ct  has  this  experiment  had 
on  th.:  normal  organizational  routine  at  PSS  with  respect  to 
the  interaction  between  procurement,  specification  and  market- 
ing personnel?  One  of  the  more  important  elements  of  the 
experiment  in  this  area  was  the  formation  of  a team  with 
members  from  each  of  the  areas  present,  with  clear  <mt  authority 
to  make  decisions  without  resorting  to  the  usual  chain  of  com- 
mand process.  Agency  impact  evaluation  will  focus  on  this 
process  to  determine  both  its  effectiveness  and  probability 
of  being  retained.  • 

From  the  commercial  impact  vievrpoint,  ETIP  is  interested  in 
determining  what  effect  the  incentives  used  in  this  experiment 
(definite  quantity  and  sliding  scale)  actually  had  on  suppliers* 
decisions  whether  to  participate  at  all  in  the  procurement  and 
more  specifically,  what  level  of  developmental  effort  to  put 
into  the  bid.  Clearly  there  is  an  interrelationship  of  forces 
between  pending  industry  standards  for  noise  and  safety,  and 
the  incentives  used,  and  to  the  extent  possible,  these  forces 
must  be  sorted  out  in  answering  the  question  "What  caused  the 
supplier  to  react?"  Finally,  a decision  must  be  reached  on 
whether  to  continue  this  experiment  in  future  procurement 
cycles  to  give  industry  more  time  to  develop  innovative  products. 
An  essential  ingredient  in  this  decision  will  be  feedback  from 
industry  regarding  the  experiment  in  general  and  the  particular 
incentives  used.  Refinements  in  future  cycles  would  come  directly 
from  this  feedback. 
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HWIAGEfCWT  BY  OBJECTIVES:  COWTRACTOK  WLOYffWT  COMPLI/WCE 


Roseaary  E.  Howard 
Equal  Earioyient  Specialist 
Defense  Logistics  Agency 
Caaeron  Station 


INTROOUCnCN 


I ^ ’ -"jI  .i  t V-  , 

i . V,  *4  i 

V,  . -Jj: 

i'  l •>.  j 

r • - - V'  - -c:-, \z-  'V  = 
■ ■-  - 


'/C-J  . ^ I \ 


.-ii:  »• 


^9^3,  a wegDrandus  of  significant  1a>11cat1on  was  sent  to  Secretarv 
^1*®  Honorable  Elliot  L.  Richardson.  That  wenorandui,  signed  by  the 
Rodent  of  the  Uilted  States.  rcx)uested  the  Secretary  of  Defense  (L  well  as  the 
hea*  of  20  o^r  Executive  Departents.  and  Agencies  who  received  Identical  annranduBB) 
if— f?  ?** .rf*“J**  "**1**  ^ varlOTB  prograas  and  activities  under  his  direction 

*"  President  requested  Secretary  Richardson 

to  ^velop  • »st  of  Bajor  goals  and  objectives  which  would  be  pursued  Ytf  the  Departaent 
of  tenons®  (DOD)  during  the  coning  year.  The  President  enphasized  results,  long-tem 
goal  setting  and  short-tera  objective  achlevenents,  and  advised  that  fnm  t1i«  to  tine 
*L**ffi?  review  the  progress  of  000  ^ard  achieving  these  objectives.'  By  June  1973, 
^ Managenent  and  Budget  (0»),  under  the  direction  of  the  Honorable  toy  Ash. 

^ overall  direction  and  coordination  of  this  effort 
Objecti v«  ^**^0^'*  Branch.  This  nanagerial  direction  becaae  known  as  Kanageaent  by 

Zero-base  budgeting  has  been  called  the  descendant  of  WO.  The  Govemient 
tmoRiy  and  Spen^ng  tofom  Act  of  1976,  requires  every  govemnent  organization  to 
waluate  and  review  all  prograas  and  activities  on  a basis  of  output  or  pei^raanoe,  as 
to  Increase  analysis.^  Therefore.  Ninagenent  by  Objectives  (WO)  Is 

progran  should  go.  how  to  get  there,  and  In 
‘'“^9®«n9  Is  the  approach  to  use  to  detervlne  If  the 
f**”^'*^  are  efficient  and  effective,  and  If  they  should  be  ellninated  or  re- 

duced to  fund  higher  priority  new  prograas,  or  to  reduce  the  current  bu^t. 

r—i  ^ P“n>ose  of  this  article  Is  to  show  the  applicability  of  WO  to  the  Contractor 
^loy^t  Coopllance  function  of  the  Departaent  of  Defense,  and  how  Its  resulte  can 
be  used  to  support  the  newer  concept,  zero-base  budgeting.  The  background  of  the 

shoOT,  foiled  by  a description  of  the  WO  technique  employed 
.P*®  conclusion  will  focus  on  the  wty  In  which  WO  data  and  design 
can  be  supportive  of  the  zero-base  approach  to  budgeting.  ^ 

BACKGROUND 

rd  E^lpywent  Opportunity  Is  the  law.  Dlscrtalnatlon  Is  prohibited  by  the 

Civil  Righte  Act  of  1964,  and  Executive  Order  11246.  as  amended.  The  Order  Is  adminis- 
tered by  the  Secretary  of  Labor,  through  the  Director,  Office  of  Federal  Contract 

D4  k Honorable  Elliot  L.  Richardson,  Secretary  of  Defense,  from 

Richard  Nixon,  President,  April  18,  1973. 

Peter  A.  Pyhrr,  "The  Zero-Base  Approach  to  Goverment  Budgeting,”  Public 
M»1n1strat1on  Review,  January/February  1977,  * 
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CMDllanee  Prograre  (OFCCP),  Department  of  Labor.  Described  In  Title  41,  Code  of 
Federal  Itegulatlons.  ‘The  Order  prohibits  discrimination  because  of  race,  color,  religion 
sex,  or  national  origin,  and  requires  affirmative  action  to  ensure  equality  of  opportun- 
ity in  all  aspects  of  eiployment  by  all  Federal  govenunent  contractors  and  subcontractor, 
and  by  all  contractors  performing  work  under  federally-assisted  construction  contracts.  ■» 


The  Department  of  Labor,  in  turn,  has  assigned  responsibility  to  several  Fefcral  ] 
Govemnent  co^>11anoe  agencies*  bv  major  Industry  groups*  using  a Standard  Industrial 
aassification  UlC)  code  system.'  A ca^>lianoe  agency  is  one  designated  to  conduct 
coBrehensive  reviems  of  contractors'  e^iloyment  practices,  and  to  undertake  such  other 
responsibilities  in  connection  vrith  the  a<Mn1stration  of  the  Order  determined  by  the 
Departnent  of  Labor  to  be  appropriate.  The  Department  of  Defense  is  one  of  tmelve 
campliwoe  agencies.  To  accxm^lish  this  mission,  the  Executive  Directorate.  Contractor 
Employment  Co^>lianoe.  Defense  Logistics  Agengr,  through  nine  regional  offices,  conducts 
an  extensive  program  of  co^iliance  reviws.®  The  geographical  spread  of  region 
responsibilities  is  illustrated  In  Table  1.  The  major  Industry  groups  ibridiich  the 
Department  of  Defense  has  conpliance  responsibility  are  illustrated  by  SIC  codes,  in 
T*le  2.  - • - 

Althou^  the  purpose  of  the  Order  is  to  promote  and  ensure  equal  opportunity  for 
all  persons  enployed  or  seeking  employment  with  government  contractors,  both  those 
contractors  who  provide  supplies  and  services*  and  those  idio  perform  under  federally* 
assisted  construction  contracU.  this  WO  appvMch  considers  only  the  enpiqyees  for 
supply  and  service  contractors.  ^ i ' >> 

— > i- 

A sl9iif1cant  difference  in  administering  the  Contractor  EPployment  Coiqillance 
program  over  others  in  contract  administration  is  the  origin  of  the  contract.  By  using 
the  SIC  code  method  for  assigning  conpliance  responsibility  to  an  agency.  DOD  Is  con- 
cerned with  coapanies  who  may  hold  contracts  with  other  Federal  government  agencies,  and 
not  necessaHly  with  DOO  buyers.  For  exa^>le.  SIC  code  027  signifies  Printing  and 
Publishing  as  a major  industry  group.  Therefore,  any  government  agenqr  contracting  its 
printing  and  publishing  work  looks  to  DOO  for  coopliance  responsibility.  Similarly, 
thera  are  000  contracts  for  tdiich  conpliance  responsibility  is  held  by  another  agency. 
kt  exMple  is  wholesale  groceries,  SIC  code  514,  a major  industry  group  tdiere  the  ' .. 
Department  of  Agriculture  has  responsibility,  regardless  of  the  voluae  of  contracts 
made  by  DOD.  For  iniformity  in  obtaining  data  on  current  contracts,  across  Agency 


^itle  41.  Code  of  Federal  Regulations.  Chapter  60.  Part  1.1.  Government  Printing 
Office,  JljIylstS: 

^Standard  Industrial  Classification  (SIC)  code  is  a numeric  coding  system,  estab- 
l.si'ed  by  the  Office  of  Management  and  Budget,  used  to  describe  major  Industry  groups, 
mid  enployed  by  the  Department  of  Labor  in  assi^iing  responsibility  to  co^iliance 

agencl^^  purpose  of  the  conpliance  review  is  to  determine  if  Ue  prine  contractor 
or  subcontractor  maintains  nondls criminatory  hiring  and  e^iloynent  practices,  and  is 
taking  affirmative  action  to  ensure  that  applicants  are  enployed,  and  that  enployees  | 
are  placed,  trained,  up-graded,  promoted,  and  otherwise  treated  during  employment 
without  regard  to  race,  color,  religion,  sex,  or  national  origin.  It  consists  of  a 
comprehensive  analysis  and  evaluation  of  each  aspect  of  the  aforementioned  practices, 
policies,  and  conditions  of  employment.  Where  required,  reconmendations  for  appropriate 
sanctions  are  made. 
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mcurcment  lines.  Information  Is  received  on  a monthly  basis  from  Dui  and  Bradstrest. 
The  data  provided  to  each  of  the  nine  regions  cited  above  Include  the  name  of  the 
contractor,  contract  nuober,  location,  dollar  value  of  the  contract,  nimter  of 
employees  at  each  facility,  SIC  code,  and  the  name  of  the  contracting  agency.  A 
contractor  universe  Is  a key  element  In  the  Contractor  Employment  Co^llance  program, 
and  represents  the  total  ninber  of  contractors  for  uhlch  DOD  has  a coipllance  responsi- 
bility. Only  those  contractors  holding  current  contracts  for  at  least  $M,000,  and 
having  at  least  50  ei^loyees,  are  Included  In  the  universe.  By  the  end  of  FY  76, 

universe  Included  35,600  companies  holding  contracts  for  supplies  and 

services#® 


Each  year,  the  allocated  resources  penelt  only  8.000  compliance  reviews  of 
contractor  facilities*  of  the  3Sp600  In  the  universe.  A long-term  goal  of  the  Contract- 
or  ^jo^nt  C0ii|>1  lance  program  Is  to  wrk  toward  the  net  Increase  In  the  nunber  of 
qualified  vtonat  and  minorities  employed  In  contractors*  work  forces.  The  minimum  rate 
Is  a IX  net  Increase  per  year.  Therefore*  the  equal  opportunity  specialist  personnel 
located  In  the  nine  regions  must  schedule  and  conduct  compliance  reviews  for  those 
contractors  located  In  areas  where  locally  and  nationally  developed  enployment  statistics 
Show  women  and  minorities  exist.  These  statistics  show  the  availability  of  eiployees 
on  a Standard  Itetropolltan  Statistical  Area  (SM5A).  or  geographical,  basis.? 


The JJBO  con^t  was  Introduced  1m  the  Contractor  Ei^iloynent  COnpl1?nce  progroi  In 
wto^  1976,  to  deal  with  a problem.  The  problem  Is  how  to  establish  a method  for 
wtermlning  short-teim  objectives,  and  measuring  program  results  toward  meeting  the 
long-term  goal,  I.e.,  the  eiployment  of  qualified  women  and  minorities  at  all  levels 
“"tractors*  work  forces.  In  relation  to  their  availability  In  geogr^lcally  sUted 


M«AGE»CNT  BY  OBJECTIVES  r.- ^ 

At  a meeting  of  the  Directors.  Contractor  E^iloyment  Compliance,  held  during 
August,  1976,  In  Alexandria,  Virginia,  the  major  program  goal  was  discussed,  I.e.,  to 
msure  equal  opportunity  In  enployment.  Inasmuch  as  there  was  no  method  to  forecast 
the  rate  of  progress  toward  meeting  the  long-term  goal*  a research  project  undent 
^or  to  the  meeting  was  Introduced.  The  project  was  developed  at  the  Headquarters* 
tefense  Logistics  Agency  In  the  Executive  Directorate,  Contractor  Employment  C^llance. 
It  wu  entitled  Application  of  Hanagement  by  Objectives  to  the  Contractor  Envloyroent 
ao|>11wce  Program,  md  It  was  subsequently  Implemented  In  the  nine  regions,  on  1 Octo- 
ser  1976.  It  was  selected  because  It  provides  for  collective  reasoning  by  all  persomel* 
the  scheduling  of  facilities  to  be  reviewed  which  appear  to  have  the  most  opportunities 
for  hliing/upgrading  enployees*  and  an  auditing  of  the  results  of  the  compliance  program. 
The  auditing  was  designed  by  using  tht  Agency  management  reporting  system.  Automated 
Hanagement  Information  System  (AMIS). 8 


^Contractor  Employment  Cooml lance  Equal  Opportunity  Report.  Defense  Supply  Ager.qr* 
Noveiher^  1976.  p.  6. 

standard  Metropolitan  Statistical  Area  (SffiA)  Is  a numeric  coding  system,  estab- 
lished by  the  Bureau  of  Standards,  Department  of  Comnerce,  for  uniform  use  by  Federal 
governmt  agencies,  to  designate  large  metropolitan  areas  In  the  US.  There  are  208  SMSAs. 

BAutomated  Hanagement  Information  System  (AMIS)  Is  the  management  reporting 
process  used  by  the  Contractor  Employment  Coopllance  program,  with  the  Input  of  data  by 
the  nine  regions  to  a data  bank  at  the  Headquarters.  The  purpose  of  AMIS  Is  to  provide 
historical  and  current  Information.  This  Information  1<  the  basis  upon  which  proper 
management  decisions  can  be  made  In  directing  the  program. 
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Oita  far  the  period  FY  74  . 75,  and  76  were  used  to  show  the  percentage  trend  for 
net  Increases,  or  decreases,  in  the  enploynent  of  woaen  and  minorities  in  contractor 
wort  forces  where  region  personnel  had  already  wde  oo^>1iance  reviews.  These  data 
are  illustrated  for  each  region  on  Table  3.  by  oercentage.  The  short-term  objectives 
to  be  established  each  year  for  a five  year  period,  I.e.,  FY  77  throu^  FY  81,  are 
whole  nimhers  and  percenUges,  and  will  be  measured  against  the  actual  Increases  shown 
In  forthcoming  AMIS  reports.  WO  Is  a five  year  plan,  desisted  In  four  parts  to  focus 
on  the  annual  results  of  the  Contractor  Eoployment  Compliance  program  activities  through- 
out the  nine  regions.  The  four  parts  of  the  plan  follow: 

1.  Identification  of  the  Responsibility  l-g**1«  ^ * decentralized  core  program, 
is  directed  toward  a 'decision  wit"  coeposed  of  region  personnel,  with  specific 

geographical  responsibility  and  mission.  Therefore,  the  level  of  operation  for  the 
WO  program  exists  at  the  DCASHA.:'  The  decision  unit  consists  of  assisted  personnel, 
resources,  mission  mid  authority.  Within  each  OCASIK  there  are  Standard  Wtropolltan 
Statistical  Areas  (SWAs)  which  were  mentioned  earlier  In  this  article.  Thoy  are  keyed 
to  metropolitan  areas,  as  the  name  Implies,  and  are  used  with  the  data  furnished  by  Dun 
and  Bradstreet  In  the  contractor  Identification  phase,  which  are  desimied  cn  a geograph- 
ical basis. 

A 

The  OCASNAs  are  concerned  with  enploynent  categories*  To  hire  men  and  orlnorit* 
les  In  contractors'  wort  forces  Is  not  enough  to  meet  the  Intent  of  the  Executive  Order 
and  subsequent  regulations.  They  oust  be  hired  and  upgraded  to  be  represented  at  all 
levels  of  the  work  forces,  I.e.,  to  higher  levels  where  their  qualifications  and 

Indicate  they  can  be  employed.  For  this  reason,  the  MBO  prograa  Is  based  on 
setting  objectives  to  be  reached  for  each  of  the  nine  EEO-1  categories  In  a given 
geographical  area.  An  EEO-1  category  describes  a najor  eiployiaent  group  expected  to  be 
foun^.  In  a contracto'-'s  work  force.  There  are  nine  EEO-1  categories  recognized  In 
Industrial  personnel  systens:  Officials  and  Managers,  Professionals,  Technicians, 

Sales  Workers,  Office/Clerfcal  Personnel,  Craftspersons,  Operatives,  Laborers,  and 
Service  Workers. 

2.  MeUro^loyy  and  the  lise  of  the  "Decision  Package."  The  enaployment  objectives 
for  woaien  and  alnoritles  are  tied  to  their  availability  in  a given  geographical  area, 
the  SMSA,  regardless  of  the  type  of  supply  or  service  the  contractor  provides.  The 
AMIS  data  are  sorted  by  EEO-1  category,  by  SMSA,  to  establish  a aethod  €6r  typing 
eeploynent  levels  to  availability  statistics  provided  by  govemaent  sources.  Each 
OCASMA  Is  responsible  for  gathering  the  availability  statistics. 

To  confute  the  objectives,  whole  number  and  percentage,  a sanple  chart  can  be 
used.  Table  4.  By  using  the  availability  statistics  discussed  above,  there  are  three 
remaining  factors: 

a.  Opportunities  Index.  This  nmrber,  based  on  past  AMIS  data  for  the  period 
FY  74,  75,  76,  Is  an  average  of  the  three  year  period  of  the  total  nuiiter  of  new  hires 
and  promotions  idilch  occurred  In  contractor  wort  forces.  The  opportunities  Index 
•^P*^*uts  the  chances  the  contractors  have  had  to  place  or  upgrade  women  and  minorities 
In  the  reporting  period.  "Hiese  data  are  Input  into  region  computer  terminals  each 
time  a compliance  review  is  completed,  and  are  sorted  by  EEO-1  category,  and  by  SMSA. 


^Defense  Contract  Administration  Services  Management  Area  (DCASNA)  is  a 
sub-region  unit.  Several  OCASMAs  form  a region. 
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b.  Labor  Force  Percentage.  These  percentages  Illustrate  the  population  between 
the  ages  of  16  and  60,  considered  enployable.  who  are  women  and  minorities.  They  are 
established  by  SMSA,  not  by  EEO-1  category,  and  can  be  obtained  by  each  XASMA  from 
the  Bureau  of  the  Census,  US  Oepartiaent  of  Cooioeixe.  The  labor  force  percentage  1$ 
the  ultimate  long  range  goal  of  the  Contractor  Employment  Compliance  program,  that  Is 
to  s^  idien  women  and  minorities  are  e^)loyed  In  relation  to  the  labor  force  percentage 
they  represent. 

^ Objectives.  The  objectives  to  be  set  by  DCASMA  personnel,  under  the  direction 
of  region  managers,  are  the  anticipated  net  changes  of  employees  In  each  EEO-1  category 
idilch  are  expected  to  be  achieved  through  the  conpllanoe  review  process.  The  objectives 
•dll  be  shown  by  SM5A,  and  established  for  each  of  the  five  years  of  the  WO  plan. 

• To  compute  the  objectives  the  DCASNA  personnel  use  three  criteria:  availability 
statistics,  opportunities  Index,  and  the  labor  force  percentage.  The  objectives  are 
^t  to  correct  those  EEO-1  categories  where  women  and  minorities  are  not  represented 
In  proportion  to  their  availability  In  the  respective  SHSA.  According  to  standards  set 
ly  the  Headquarters  managers,  no  less  than  a It  Increase  per  EEO-1  category,  per  year, 
•dll  be  considered  acceptable.  .r  r-  .r  t 

Once  the  objectives  are  established  by  the  XASMA,  and  the  region  personnel  have 
revleired  and  approved  the  proposed  activity,  the  resources  can  be  applied.  Data 
resulting  from  the  coq[)l lance  reviews  must  be  transmitted  through  the  computer  terminal 
to  the  Headquarters,  as  required  1m  the  regular  management  reporting  system.  AHIS. 

3.  Review.  The  third  of  the  four  part  WO  program  Is  a review  process  conducted 
at  the  region  and  headquarters  levels  to  Insun^  the  program  Is  txiiiclng  effectively  and 
efficiently  toward  the  long-range  goal.  Review  of  AMIS  reports  must  be  mde  at  least 
^-annually  to  measure  the  Impact  of  the  work  reported  by  the  OCASMA  operational 
level.  Review  takes  Into  consideration  the  objectives,  resources  allocated,  and  success 
of  DCASKA  personnel  while  conducting  their  co^)11ance  reviews.  The  accouitablllty  of 
all  DCASHA  team  meiribers  Is  significant  because  the  success  of  the  program  depends  on 
their  seriousness  of  purpose  as  they  make  a comitment  In  setting  objectives,  and  work 
toward  reaching  the  objectives. 

4.  Evaluation.  Problems  encountered  nst  be  resolved.  For  exaupte,  use  of 
statistics  according  to  the  WO  formula  Is  described  easily  on  paper.  In  practice, 
however,  the  availability  of  the  actual  statistics  may  vary  among  SMSAs.  In  such  a 
^e.  the  DCASMA  personnel  may  establish  data  from  their  own  conmunlty  resource  files 
before  proceeding  with  the  actual  objectives.  Headquarters  personnel  examine  the 
effectiveness  of  the  WO  program.  This  evaluation  process  requires  an  effective 
coanunl cation  system  •dthin  the  entire  Contractor  E^loyment  Compliance  program.  The 
comnunlcatlon  expedited  resolving  problems  as  they  are  encountered.  Instead  of  waiting 
for  formal  reporting  periods. 

AMIS  data  arc  furnished  by  the  headquarters  to  the  region  each  six  months.  In 
turn,  the  region  studies  the  results  depicted  on  the  reports  and  sends  the  material  on 
to  the  OCASMAs.  The  data  serve  as  feedback  to  everyone,  and  serve  as  a basis  to  Iden- 
tify areas  which  need  Improvement.  The  accountability  factor  eidbodled  in  the  personnel 
performance,  as  team  members  work  Individually  and  collectively  to  reach  short-term 
objectives.  Is  the  most  significant  part  of  the  WO  program. 

The  WO  five  year  plan  focuses  on  results,  and  provides  a data  base  upon  which 
^dslws  can  be  made.  For  example.  If  there  Is  a greater  likelihood  for  a net  Increase 
in  eiployment  to  exist  at  one  SWA  over  another,  resources  can  be  applied  to  support 
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that  effort.  This  data  base  Is  related  to  the  zero-base  budgeting  concept.  Its  review 
and  evaluation  process,  with  cost/benefit  focus,  tends  to  support  a zero-base  budget 
preparation.  Inasmuch  as  only  22  X of  the  total  number  of  contractors  can  be  reviewed, 
the  benefits  received  from  the  application  of  resources  In  each  DCASNA  are  l^iortant 
on  a cost/benefit  basis. 

CONausiON 

There  are  two  questions  raised  In  the  Management  by  Objectives  program  which  are 
the  focus  of  the  zero-base  budgeting  process: 

1.  Are  the  current  activities  efficient  and  effective? 

2.  Should  current  activities  be  eliminated,  or  reduced  to  fund  higher  priority 
new  programs,  or  to  reduce  the  current  budget? 

Zero-base  Is  an  approach,  not  a fixed  procedure  or  a set  of  foram  to  be  applied 
uniformly  from  one  organization  to  the  next.  The  four  parts  of  the  160  program 
applied  to  Contractor  Employment  Coiqillanoe  correspond  to  the  four  basic  ste^  of  the 
zero-base  approach: 

1.  Identify  decision  wits 

2.  Analyze  each  decision  wilt  In  a decision  package 

3.  Evaluate  and  rank  all  decision  packages  to  develop  the  appropriations  request. 

4.  Prepare  the  detailed  operating  budgets,  reflecting  those  decision  packages 
approved  In  the  budget  appropriation. 

Zero-base  budgeting  calls  for  a five  year  plan  for  annual  review  and  evaluation 
of  programs,  on  a cost/benefit  basis.  Including  the  setting  of  objectives.  Program 
evaluation,  operational  decisions,  and  budget  preparations  are  subsequent  sequential 
steps. 

The  decision  packages,  developed  by  DCASMA  personnel,  can  be  analyzed  In  the 
zero-base  approach  In  light  of  six  criteria: 

1.  The  purpose  and  objectives  of  the  Contractor  Employment  Compliance  program. 

2.  The  descriptions  of  actions  taken  (what.  how). 

3.  The  cost  and  benefit  of  the  orogram. 

4.  The  workload  and  performance  measures. 

5.  Alternative  means  of  accoopllshing  the  objectives. 

fi.  Levels  of  effort,  l.e. , what  benefits  are  accrued  for  each  l..vel  of  funding, 
at  the  responsibility  levels. 

The  evaluation  and  ranking  of  the  decision  packages  enables  managers  to  direct  the 
program,  analyze  the  program  at  all  stages,  and  apply  resources  to  accomplish  the  short- 
term objectives.  The  WO  data  are  designed  to  allow  managers  to  determine  if  the 
current  activities  cited  In  question  one,  above,  are  efficient  and  effective.  These 
data  support  managerial  decision  for  continuing,  eliminating  or  reducing  the  program 
activities  of  OCASHAs,  In  response  to  question  two,  above. 

Zero-base  means  the  evaluation  of  all  program  activity.  The  evaluation  can  lead 
to  a complete  rethinking  and  redirecting  of  the  program.  Both  Management  by  Objectives 
and  Zero-base  Budgeting  require  a great  deal  of  effective  admi.il  strati  on,  coMunIcatlon 
and  training  of  all  personnel  Involved  In  the  analysis.  These  concepts  are  management 
and  budgetary  improvement  efforts  which  may  require  several  years  to  reach  full  utili- 
zation and  effectiveness. 

It  Is  my  conclusion  that  WO  Is  being  applied  usefully  to  the  Contractor 
Eoployment  Ccapllance  program.  It  Is  applicable  to  other  Compliance  Agencies  of  the 
Federal  Government  and  should  considered  by  the  Department  of  Labor  as  a uniform 
requirement.  The  fact  that  Management  by  Objectives  supports  the  newer  concept  of 
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A»»tgncaent  of  Cosnpllance  Responsibility  by  .Itapdard  Industrial 
; Classification  (SIC)  Code  • , 


,r.:; 


f 


• ^ OPOCP  Order  Ro.  1 (Revised),  issued  •<y  the  Ivpartinent  of  labor, 
assigns  coopliance  responsibility  to  et-  -J* *  o.‘  the  compliance 

t agencies  according  to  Standard  Industrial  Classification  (SIC) 

codea.  These  SIC  codes  identify  the  sajor  activity  of  a 
_ coa|>any  or  its  irllvldual  facilities.  Under  this  system,  DOO 
has  been  assigned  compliance  responsibility  for  the  folloalns 
SIC  codes:..  , * 

,J - J.  . c.  r*.,,  ■ >■  5 i— »*— -^7— — 

Supply  and  Service  Contractors  " ' -.  f*  ' ■ 


r ♦ 

*■  ‘ f- 


[ 22 

• 27 

33 

. ^—34 

^ 352 

• 359 

36 
- 372 
376 
39 
, 501 
504 
506 
^ 507 
) 508  • 
509  - 
513  • 
525  - 
527  - 

551  . 

552  - 

555  - 

556  - 

557  . 
559  - 

56  - 
739  - 


•/ 

^•4 


■ Textile  aill  products 

- Apparel  and  related  products  » 

- Printing  and  publishing  industries  " ^ 

- leather  and  leather  products  * 

• Prlnary  netal  Industries 

• Fabricated  netal  products  J 

• op  to  and  including  f*  ' ^ 

Ronelectricral  machinery  (except  engines  and  turbines  - 351) 

• Motor  vehicles  and  e<|uipfiient  . 

» Aircraft  and  parts  ' *'  , h 5 

' Guided  missilesg  space  vehicles,  and  parts  V A 

• Miscellaneous  manufacturing 

Ibtor  vehicles,  automotive  parts,  and  supplies  (wholesale) 
Sporting,  photographic  and  hobby  goods  (wholesale)  - 

Electrical  goods  (vhc^lesale) 

Hardware,  plumbing  equipment,  and  supplies  (wholesale) 
Machinery  equipment  and  supplies  (wholesale) 

Miscellaneous  durable  goods  ^ ^ 

Apparel,  piece  goods,  and  notions  ' j 

Hardware  stores  (retail)  k 

Mobile  home  dealers  (retail)  ' - 

up  to  and  including  * ^ 

New  and  used  retail  motor  vehicle  dealers  ' 

Boat  dealers  (retail) 

Recreational  and  utility  trailer  dealers  (retail) 

Motorcycle  dealers 

Autooiotive  dealers,  not  elsewhere  classified 
apparel  and  accessory  stores 
Miscellaneous  business  services 
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A1^A^TA 

Kinorities 

754 

20.4 

1062 

21.6 

1699 ' 

21.4 

i 

Vocen 

39.8 

39.1 

?9.3 

‘ 

i 

- 

IXTON 

Minorities 

466 

07.7 

030 

07.6 

543 

06  .S 

• 

Women 

27.4 

26.1 

1 

. 

cacACO  . 

Minorities 

465 

14.5 

604 

17.0 

541 

14,4 

- 

Voncn 

23.4 

26.1 

22.9 

/ 

(LEVELAIID 

Minorities 

571 

15.0 

702 

15.0 

844 

13.7 

Women 

17.1 

16.9 

19.4 

1 

t 

DALLAS 

Minorities 

674 

26.1 

738 

27.2 

723 

23.9__ 

31.8 

Women 

28.9 

32.5 

LX  ANGELES 

Minorities 

491 

17.1 

591 

U.8 

763 

16.2 

♦ 

Women 

25.0 

22.8 

22.7 

Vt4  rORX 

Minorities 

343 

17.4 

649 

19.6 

426 

Women 

32.0 

28.6 

29.1 

FKir.ADEXFKIA 

Minorities 

278 

09.3 

349 

10.2 

414 

09.6 

1 

Women 

16.9 

24.0 

21.0 

““  ( 

. • 1; 

sr.  LOUIS 

Minorities 

331 

08.9 

485 

09.1 

509 

09.2 

3 

{ 

<• 

i 

! 1 ' 

Women 

23.0 

24.8 

22.4 
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John  V.  Hal sh  ‘ •• 

Department  of  Defense  ' ’ V." 

Air  Force  Office  of  Scientific  Research  (AFSC)  - :lr‘  ; 

I 10'  - Si  V 


1.  The  DeparCaent  of  Defense  (DoD)  la  reaponsible  for  assuring  chat 
the  scientific  research  necessary  to  accoisplish  ita  lassioa  is  given 
adequate  support  and  that  it  is  Che  policy  of  the  DoD  to  conduct  a ' 
broad  and  continuing  basic  research  progran  to  provide  fundanental 
knowledge,  with  enphasls  on  that  related  Co  the  needs  of  the  DoD. 

2.  Until  1958»  contxncts  vere  the  only  cieens  by  vhich  the  DoD  could 

arrange  for  research  services  or  support  at  either  educational  insti* 
tutions  and  other  nonprofit  organisations  or  coooercial  concerns. 
Congress  had  recognised  the  need  for  a sinple  economical  and  effective 
instrument g unencumbered  by  many  contract  provisions , which  would 
allow  certain  freedom  and  creativity  to  the  scientific  coasamity.  In 
that  yearg  a law  was  enacted  which,  for  the  first  time,  gave  the  Dob  * 
authority  to  expend  funds  through  grants  for  support  of  scientific 
research.  - * .... 

3.  'Sectloa  1 of  that  Statute,  P.L.  85-934  provides:'  * 

"That  the  head  of  each  agency  of  the  Federal  Goveriment,  authorized 
to  enter  into  contracts  for  basic  scientific  research  at  nonprofit 
institutions  of  higher  education,  or  at  nonprofit  organisations 
whose  primary  purpose  is  the  conduct  of  scientific  research,  is 
hereby  authorised,  where  it  is  deemed  to  be  in  furtherance  of  the 
objectives  of  the  agency,  to  make  grants  to  such  institutions  or 
organisations  for  the  support  of  such  basic  scientific  research.** 

4.  Section  2 provides: 

• * r'  ; 

''discretionary  authority... • to  vest  in  such  institution  or 
organization....  title  to  equipment  purchased  with  such 
grant  or  contract  funds'* 

5.  While  the  authority  provided  by  P.L.  85-934  is  quite  broad  and 
general,  a series  of  implementing  directives  have  been  issued  by  the 
Department  of  Defense  imposing  certain  limitations  and  restrictions 
on  the  use  of  grants  by  the  military  departments. 


The  directives  provide  coverage  for: 

a.  Use  of  Grants  - A grant  will  be  limited  to  the  sur*x>rt  of 
those  research  (authorized  by  P.L.  85-934)  projects  which  meet 
relevant  research  requirements  of  the  Department  of  Defense. 


« Grant  - Prior  to  awrd.  Grantee  nuat  have  caviled 
with  the  Civil  Rights  Act  of  1964,  through  Letters  of  Assurance,  and 

4 ff«  4 — - ^ ^ ^ m 9 


certain  environsKntal  considerations. 


^viL 


*•  ot  Grantee  - Grantee  aust  be  a qualified 

educational  institution  or  nonprofit  organization  whose  priaarv 
purpose  is  the  conduct  of  research. 


**•  to  Equipaent  - Title  to  equipaent  purchased  with  research 

fun^  shall  be  vested  in  the  grantee  organization  as  follow: 

eeeentially  the  sasie  procedure  as  for  contracts  under  ASPR) 


(1)  For  each  itea  of  equipaent  having  an  acquisition  cost  of 
less  than  $1,000  title  shall  be  vested  autosntically  in  the  grantee 
organization  upon  acquisition. 

■ ■ • J--.-..  1 . 

Iteas  of  equipaent  having  an  acquisition  cost  of 
$1,000  or  aore  title  shall  be  vested  in  the  following  wnner: 


^itle  to  Covemaent  property  for  perfoi 
and  purchased  with  research  funds  will 
upon  acquisition. 


■nee  of  work  under  the  grant 
■lly  vest  in  the  grantee 


If  there  is  a genuine  basis  for  ezpectlng  that  a Goveroaent  need  will 
exist  at  the  end  of  the  research  period,  title  will  vest  in  the  grantee 
u^n  acquisition  subject  to  a right  of  the  Govemaent  to  direct  transfer 

^ Gove^^ot  to  • third  party  authorized  to  receive 
it  within  4 aonths  after  coapletion  of  the  grant. 


If  the  contracting  officer  detemines  that  vesting  of  title  in  the 
grantee  would  not  be  in  furtherance  of  the  OoD's  research  proeraas 
title  will  vest  in  the  Govemaent. 


e.  Contents  of  Grant  - Grant  agreeaents  shall  be  brief  and  contain 
only  those  provisions  »diich  are  required  by  Statute  or  are  necessary 
for  the  protection  of  the  fundamental  interests  of  the  DoD.  Provision 
•hall  be  made  for: 


(1)  Maintenance  of  adequate  records  to  document  the  actual 
amount  of  any  participation  and  to  determine  whether  grant  funds  are 
properly  expended. 

(2)  Provisions  for  appropriate  patent  and  data  rights. 

(3)  Provisions  for  revocation  of  the  grant. 

(4)  Provisions  f..r  the  grantee  organization  to : 
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(*)  Obtain  approval  from  the  grantor  agency  before 
changing  the  principal. investigator. 

• j'J 

(b)  Keep  the  grantor  agency  inforaed  of  any  desired  ’ 

■ajor  deviation  froa  the  planned  work  and  progress  under  the  grant. 

' ' I'c)  Furnish  results  of  all  research  to  the  grantor  agency. 

f.  Cost  Sharing  - Coat  sharing  (participation)  in  the  support  of 
research  shall  be  encouraged,  except  when  the  organization  has  little  ‘ ‘ 
or  no  non-Federal  sources  of  funds  fron  which  to  nake  a cost  contribution. 

1 

CKAHT  COHTKOLS  ‘ Jss',;  {Si 

• • * i r . ■?*  ;>•/  S'!  • f ^ 

1.  Ulile  aany  of  the  checks  and  balances  inherent  in  cost  reiabursenent 
contracts  either  do  not  appear,  or  appear  in  a wore  relaxed  fora,  in  onr 
grants,  adequate  controls  are  provided  to  insure  that  grants  properly 
protect  the  Governaeut's  interests,  and  that  neither  the  quality  nor  » 

of  research  results  obtained  have  been  adversely  effected  by 
their  use.  To  insure  that  available  funds  are  eaployed  in  such  a wanner 
as  will  be  aost  Ukely  to  provide  the  Covernaent  with  the  aaxiaia  return 
for  its  investaent,  the  need  for  certain  safeguards  and  controls  is 
obvious.  Soae  of  the  areas  la  nhich  such  need  has  been  recognized, 
and  the  extent  to  vb«ch  our  grant  procedures  satisfy  need,  are 
as  follows: 

a.  Basic  research  proposals  are  selected  for  support  without  reward 
to  the  type  of  instruaent  tdiich  will  be  used.  A srall  percent  of  the 
proposals  received  ara  ultiaately  selected.  This  selection  is  based 
largely  on  the  zeUtive  scientific  aerlt  of  the  technical  proposal  and 
the  standing  of  the  prospective  investigator,  and  the  Institution  with 
which  he  is  affiliated,  in  the  scientific  conamity.  Only  after  a 
proposal  has  been  found  to  merit  support,  based  purely  on  scientific 
and  technical  considerations  in  competition  %»ith  all  others,  is 
consideration  given  to  the  amount  of  funds  requested  and  the  fora  of 
*^*®crch  agreement  to  be  eaiplo3red.  In  such  a highly  competitive  cliaate, 
we  are  reasonably  assiured  of  high-quality  research  results  regardless  of* 
the  method  by  which  support  is  provided. 

b.  The  same  criteria  is  used  for  contracts  and  grants  in  determining 
whether  proposed  costs  are  fair  and  reasonable,  and  both  costs  are 
subjected  to  the  same  degree  of  analysis.  Hegotiations  are  conducted 

in  the  sane  manner,  and  deterninations  as  to  reasonableness  are  equally 
substantiated,  regardless  of  the  type  of  research  agreement  to  be  used. 

c.  A number  of  essential  controls  are  provided  by  the  grant 
docisient,  itself.  Among  these  are  the  following: 
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(1)  The  grantee's  research  proposal  is  incorporated  by 

reference*  The  grantee  agrees  to  adhere  thereto  in  the  conduct  of 
the  research.  While  the  nature  of  basic  research  is  such  that  the 
widest  practicable  latitude  for  its  conduct  should  be  provided,  so 
long  as  there  is  not  a departure  froa  the  objectives  forming  the 
basis  for  its  selection  for  support.  We  make  no  attesipt  to  direct 
the  manner  in  which  the  research  is  conducted,  but  do  provide 
safeguards  against  undesirable  departures  through  a system  of  t 

technical  nonitorship  and  retention  of  a unilateral  right  to 

revoke  the  grant  at  any  time.  ’ 

OO 

(2)  Grant  brochures  for  basic  research  are  incorporated  by 
reference.  They  generally  establish  the  framework  within  which  the  * -- 
research  project  will  be  conducted  and  the  grant  will  be  administered. 

It  makes  appropriate  disposition  of  such  matters  as  relate  to  patent 
rights,  rights  in  technical  data,  revocation,  security,  unexpended 
funds,  title  to  equipment,  equal  employment  opportunity,  etc. 

(3)  Reports  of  research  results,  identical  to  those  we  ' ^ 

require  under  our  research  contracts,  are  provided  for  in  our  grants. 

(4)  Reports  of  actual  expenditures  by  major  categories  of 
cost  are  required  to  be  submitted  annually  and  at  the  end  of  the  grant 
period.  Coi^risons  between  these  reports  and  the  original  estimates 
enable  us  to  determine  the  extent  to  which  cost  objectives  have  been 
met.  Selective  audits  are  made  to  verify  accuracy  of  reporting,  and 
demands  are  made  for  the  return  of  improper  expenditures.  Demonstrated 
fiscal  responsibility  weighs  heavily  in  determining  whether  to  continue 
support  of  on-going  projects  or  new  projects  at  a given  institution. 

(3)  In  their  negotiation  and  administration,  we  treat  grants 
as  a simple  form  of  fixed-price  agreement  providing* for  advance  payments. 
Once  executed,  the  grant  amount  is  a fixed-ceiling  amount.  The  grantee 
absorbs  lOOX  of  all  costs  incurred  in  the  conduct  of  the  research 
project  in  excess  of  the  grant  amount  and  returns  any  remaining 
funds  at  the  end  of  the  grant  period. 

CRITERIA  FOR  SELECTION  OF  GRANT  OR  CONTRACT  IN  AIR  FORCE 


I.  Policy  - It  is  the  policy  of  the  Air  Force  to  use  grants  for  the 
support  of  scientific  research  programs  whenever  the  contracting 
officer  considers  such  use  appropriate.  A grant  will  be  used  except 
when  any  of  the  follo%ring  factors  are  present  - in  such  cases  a 
contract  will  be  used: 

a.  When  any  portion  of  the  actual  research  services  will  be 
subcontracted  (consultants  are  not  considered  subcontractors). 

b.  When  a proposal  is  obtained  through  the  formal  solicitation  process. 


u ■, 

: ■ 


c*  When  the  type  of  work  requires  close  technical  nonitoring 
during  the  course  of  performance. 


n-'% 


U'.:  • 


d.  When  greater  fiscal  control  through  voucher ing  and  audit 


is  desired  by  the  Government. 


e*  When  the  research  requires  security  classification.  ^ 

f.  Vhen  more  than  231  of  the  funds  requested  is  for  the  ' 
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A OCMBfcRlSOM  OF  ATOSE  GRANTS  VS  CONTRACTS 

AMA  OOMTRACTS  ? . 

,OOSTIBG  Scccloa  XV  of  ASPK. 

Frlnciples  and  Procedures 


h -V-  “ ■ **•  ' • ■'  ll." 

•>vv  3 ' -iciw/V-  \ynj^t<  _■  ..  - ;ji:' 


t'  t ' ^ ’7'^  ”i 


Vfd.:,/  v:s  , - 


GKAHTS 


:l  xW  * 
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Cost  Use  of  SoctloB  ZV  of  ASPR- 

p.  Cost  Prlnclpleo  and 

^ Procedtxrea 


MSnUMEHT 

■ ;■  ‘ > 


VoluBlnooB  schedule,  special  and 
general  tenas  and  conditioas 
(25-30  pages)  which  are  required 
ASPR.  • Subject  to  frequent 


,,  ^MBgeSe 


Single  page,  incorporating 
standard  tens  of  printed 
brochure  and  grantees* 
proposal  by  reference. 


UOSK  SIATCHEMT  Required  vmder  every  contract  Rone  specifically 

describing  work  to  be  performed.  required.  Proposal  is 

incorporated  into  grant 
- by  reference,  and  serves 

as  the  work  statement. 


Ekmrr  vouchers  required  to  be  prepared 

(usually  wonthly),  and  paynent 
■ade  after  review,  approval  and 
processing  required  by  audit. 

1 

FILE  DOCUKEElATlOg  Must  include  considerable  support 
docunentation  such  as  certifi- 
cations and  representations. 
Determinations  and  Findings 
under  ASPR. 


ADMUISTRATIOM  Considerable  cooBnmicatlon  with 

contractor  is  required  concerning 
matters  where  the  C.O.'s 
authorisation  must  be  had 
before  contractor  nay  act. 


Payments  are  made  in 
advance,  on  periodic 

basis. 


Very  limited  information 
is  required. 


Little  conmunication 
necessary  for  grant 
administration. 


PBOPEKTY 


Must  secure  approval  of  facilities 
acquisition  including  a Facilities 
Acquired  or  Fabricated  clause 
in  the  contract  listing  the  iteiss 
authorised. 


No  facilities 
acquisition  approval 
required.  Listing  of 
items  in  grant  required. 


f. 
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AREA 


OOMTRACTS 


GRANTS 
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TESHIM&TION  Tentinatlon  C.O.  aosC  be  eppolnCed,  Uallatentl,  one 

CeiBiaation  claim  eubaicted.  and  page  revocation. 

lengthy  formal  settlement  -«  ~i>'  -n:.  ' ^ 

• ."t?;;-' agreement  negotiated.  ~ ■ 

,.-»  ^ i-i"!  \ ■Lr't.  IS’*';. M3,  ^ -K.5 '"s 

CLOSIHG  out  • V Great  delays  experienced  because  Closing  out  nay 

...  final  audit  is  required  on  each  " sli^»ly  be  completed  “r* 

-iC  contract  and  becauae  of  the  very  as  soon  as  final 

■ „%ri<s.^fia>i4»&5<^-r  long  period  before  many  overhead  reports  (technical, 

: 4:  rates  are  established  so  the  . fiscal,  patent)  are 

final  voucher  can  be  aubmltted.  aubmitted 
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CURRENT  CRAMT  LECISIATIOM 


1.  LegULatioD  is  curreotly  being  proposed  which  has  as  its  purpose 
V*: . t to  distinguish  Federal  grant  and  cooperative  agreesttot 
relationships  fro®  Federal  procureaent  relaxionships," 
so  as  to  "standardize  usage  and  clarify  the  Meaning  of 
legal  instnasents  which  reflect  such  relatiooships". 

This  action  is  being  pursued  through  Senate  Bill  S.  431  sponsored  by 
Senator  Chiles  and  coBM>anion  House  Bill  H-1.  1503  sponsored  by 
Mr.  Horton,  95th  Congress.  1st  Session.  LegisUtion  on  siaiUr  bills 
p*;  has  been  proposed  as  far  hack  as  1973  but  were  never  enacted.  The 
basis  for  such  legislation  stems  from  a study  by  the  Cosnission  On 
Government  ProcureKnt  reported  in  1972  under  which  a reconendation, 
referred  to  as  F«l,i«a  proposed.  The  proposed  bills  essentially  are* 
implementing  this  recommendation.  Basically  the  proposed  legislatio 
provides  that: 


••  Transactions  in  a procurement  mode  (direct  benefit  to  the 
Government)  will  be  aiade  by  contract. 

b.  Transactions  in  an  assistance  node  (not  for  the  direct  benefit 
by  the  Government)  will  be  made  by  grant  or  cooperative  agreement. 

2.  DoD  bad  difficulty  in  accepting  the  bill  as  written  since  DoD 
sponsored  research  programs  are  intended  to  obtain  scientific  results 

•'  primarily  for  Covemsient  use  to  further  its  mission.  The  use  of  a 
grant  in  the  assistance  mode,  i.e.,  for  the  direct  benefit  of  Che 
Brantee  as  defined  in  the  bill  would  not  be  appropriate  for  DoD  use. 

3.  As  wis  indicated  before.  Congress  in  1958  recognized  the  need  to 
minimize  the  administrative  burdens  on  the  military  departments, 
universities  and  non-profit  organizations  engaged  in  basic  scientific 
research  and  enacted  P.L.  85-934  particuUrly  to  authorize  the  use  of 
grants  instead  of  contracts  as  simple  instruments  to  fommlize 
agreeneats  in  basic  research. 

4.  It  should  be  noted  that  this  concept  for  supporting  basic  research 
under  P.L.  85-934  as  envisioned  by  Congress  has  not  changed.  In  fact 
the  validity  of  the  concept  has  been  greatly  reinforced  by  DoD  success 
in  acquiring  research  more  economically. 

5.  The  proposed  legislation  would  repeal  P.L.  85-934  and  would  limit 
the  use  of  grants  to  Federal  assistance  programs.  Passage  of  the  bill 
would  deprive  agencies  of  this  established  and  successful  grant 
authority  and  would  require  the  acquisition  of  research  by  less 
ecoDOoiical  means. 
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Major  Arthur  t.  King 

Headquarters,  Air  Force  Logistic  Connand 


IWTRODOCTION 


Shrinking  real  budgets  over  the  last  decade  have  iimosed 
a resource  constraint  problen  on  the  Armed  Services  without 
present  in  the  ^st-Morld  War  II  era,  given  the  assessment 
®-  military  threat.  During  this  sane  period  debate  has  ?• 

providing  an  "adequate"  defense  posture 
^ services  have  been  required  to  implement  programs 
which  did  not  contribute  directly  to  "national  defense".  The 
procurement  process  is  one  of  the  areas  upon  which  several 
STCioeconomic  programs  have  been  superimposed  for  implementa- 
tion. The  socioeconomic  proarans  alluded  to  here  include, 
but  ^e  not  limited  to,  the  Small  Business  Set-Aside  Program; 
tte  Davis-Bacon  Act;  the  Service  Contract  Act;  the  Walsh-Healy 
Act;  and  the  Action  R (a)  Program. 

Section  8 (a)  Program  has  received  much  emotional 
debate  over  the  last  several  years.  The  program  has  been 
questioned  on  judicial  and  political  grounds  and  ther«'  has 
been  considerable  controversy  over  the  efficacy  of  the  con- 
cept of  minority  capitalism  into  which  it  was  made  to  adapt. 

a(^nistered  by  the  Small  Business  Administration 
(SBA)  under  Section  8(a)  of  the  Small  Business  Action  of  1953, 
as  ameded.  Under  this  provision  of  the  act,  SBA  is  permitted 
to  contract  witJt  other  government  agencies  for  the  provision  of 
goods  and  services  and  then  subcontract  with  private  firms  to 
^ the  actual  work  "whenever  it  deems  such  action  is  necessary".^ 
This  provision  of  law  as  singled  out  to  become  the  foundation 
or  a major  portion  of  the  Federal  Government's  Minority  Enter- 
prise Program  in  1969.  It  was  the  vehicle  to  be  used  in  help- 
ing minority  owned  firms  to  become  more  viable  business  entities 
by  allowing  them  to  secure  government  contracts  noncompeti- 

"The  basic  aim  of  Section  8(a)  is  to  establish  [ninorityl 
business , concerns  that  will  become  independent  and  self-sus- 
taining in  a normal  competitive  environment,"^  Thus,  the 
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Section  8<a)  Program  is  a domestic  economic  dovelonuKnt  program 
which  has  nonwhite  minority  aroun  socioeconomic  mobility*  as 
its  principal  goal.  This  orogram,  as  all  ether  socioeconomic 
programs,  should  be  judged  on  the  basis  of  whether  its  goals 
are  worthy  of  pursuit;  whether  it  is  accomnlishing  its  goals; 
and  whether  the  benefits  of  goal  accorplishment  outweigh  the 
costs.  Within  the  context  of  the  overall  national  interest, 
the  political  decision  process  determines  which  public  oolicy 
goals  are  to  be  pursued  as  well  as  the  manner  of  pursuit. 

T|»  sheer  magnitude  of  defense  procurement  expentitures  make 
them  stand  out  as  a vehicle  through  which  other  national  goals 
CM  be  p^sued.  Indeed,  they  have  been  used  for  such  purposes 
since  the  eight-hour  work  day  was  first  extended  to  workers 
employed  by  contractors  and  subcontractors  engaged  in  Federal 
projects  in  1892."^ 

GOALS  AIID  SCOPE  OP  THE  ;a:SEARCH  * 

This  research  attempted  to  answer  two  questions:  1)  •To 
what  extent  do  socioeconomic  programs  impinge  uoon  the  pro- 
curement process?  and,  2)  What  are  the  relationships  oe- 
tween  underdevelopment  and  public  policy  activities  designed 
to  combat  i«  in  underdeveloped  countries  on  the  one  hand  and 
in  nndc:rdeveloped  sectors  of  economically  advanced  countries 
other?  The  research  addressed  itself  to  tlae  applic- 
abxlity  of  general  economic  development  theom/  to  the  economic 
development  problems  %rtiich  confront  underdeveloped  sectors 
of  advanced  economies  through  consideration  of  one  public 
^licy  program  and  one  underdeveloped  sector  — The  Section 
8(a)  Program  and  the  nonwhite  minority  group  sector^  of  the 
United  States  economy.  A basic  purpose  of  the  research  was 
the  development  of  a framework  and  rationale  for  the  evalua- 
tion  of  socioeconomic  proqr£iins. 

The  research  attempted  to  answer  the  questions  posed 
above  through  the  accomplishment  of  three  goals.  The  first 
^ these  was  the  development  of  a theoretical  framework  within 
vrhich  the  faction  8(a)  Program  and  similar  socioeconomic 
programs  could  be  evaluated.  The  second  goal  was  the 
development  of  a model  %#hich  could  be  used  to  eva3.ua te 
the  effectiveness  of  socioeconomic  nroorams.  A third  goal 
was  the  development  of  a method  of  evaluating  the  costs  of 
socioeconomic  programs  to  agenices  which  iraolenent  them. 
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rCTUODOLOGY 


This  study  was  based  in  the  analytical  framework  of  the 
theo^  of  economic  development.  This  approach  to  the  economic 
problems  of  American  nonwhitc  minority  group  roer^rs  is  neither 
novel  nor  new.  Gustav  Ranis  had  observed  the  sirolarities 
tetween  underdeveloped  countries  and  American  minority  grouos 
in  an  article  entitled  "rcononic  Dualism  at  Hone  and  Abroad*  in 
the  F^l  1969  issue  of  Public  Policy.  The  quantitative  model  • 
used  in  the  analysis  was  developed  for  this  research.  It 
translated  the  available  data  on  Air  Force  5$ection  8(a)  con- 
tracts into  variables  theoretically  irnortant  to  minority  group 
economic  lUvelopmcnt.  The  cost  of  implementation  analysis 
was  based  upon  answers  to  a survey  questionnaire  which'  was 
sent  to  Air  Force  procurement  offices.  . i 

^ T^  method  used  in  the  first  portion  of  this  research 
was  privily  descriptive.  It  investigated  the  applicability 
Of  traditional  economic  development  theor'/  to.  underdeveloped 
sectors  of  advanced  economic  by  analyzing* the  scope,  mag- 
nitudet  and  orincipal  conoonents  of  the  ninoritv  groun  economic 
underdevelopment  problenu  The  role  of  public  police 
in  eronoi^c  developnentj  the  ecomonic  development  potential 
of  minority  capitalism;  and  the  genesis  of  the  Section  8 (a) 
Program  were  then  treated.  The  second  portion  of  the  research 
provided  a quantitative  assessment  of  the  effectiveness  and 
cost  of  Air  Force  participation  in  the  Section  8(a)  Program. 

portion  of  the  research  provided  a*.*  ^umar*/  and 
conclusions. 


MINORITY  GROUP  UTTDERDKVEIjOPME^IT 


minority  group  economic  underdevelopment  problem 
which  gives  rise  to  the  need  for  govemiaentai  sociecononic 
policy  intervention  bears  more  detailed  consideration.  One 
of  the  conierstones  of  the  descriptive  analysis  was  the 
VICIOUS  circles  of  underdevelopment  argument  developed  bv 
Gunnar  Myrdal  in  the  195Ps.  According  to  this  argument# 
underdevelopment  begets  itself  through  per.sistent  interaction 
of  the  relevant  variables  all  reinforcing  each  other  to 
impede  economic  development.  In  this  manner#  the  amplitude 
of  waves  which  migh  otherwise  result  in  develooment  take-off 
are  dampened.  The  argument  is  fully  applicable  to  under- 
developed sectors.  For  example#  members  of  American  minority 
have#  in  general#  had  less  education  attainment 


301 


■t 


than  average;  which  reinforced  job  bias  in  the  form  of 
decreased  upward  employment  mobility  and  increased  un- 
employment and  underemployment;  which  decreased  the  ability 
to. purchase  suitable  housing;  which  led  to  inferior  ed- 
ucational attainment;  and  so  on.  Indeed  it  was  this 
circle  of  underdevelopment  that  the  civil  rights  laws  of 
the  1960s  sought  to  undermine.  . „ 


' -The  minority  group  sector  of  the  United  States  e<»namy 
is  also  similar  to  underdeveloped  countries  in  terms  of  *■**» 
characteristics  of  underdevelopment  which  are  prevalent  in 
it.  The  major  components  of  the  sdnority  group  underdevelop- 
ment problem  in  relation  to  averages  for  the  society  as  a 
idiole  are:  low  income  measured  either  in  per  capita  or 
median  family  terms;  a lack  of  resource  ownership, 
htmian  and  physical  capital;  and  persistently  high  unesg>loy- 
ment  and  underemployment.  These  underdevelopment  character— 
^tics  have  been  reinforced  by  interpersonal  and  institutional 
discrimination  which  relegated  minority  group  members  to 
lower  job  and  socioeconomic  classifications  based  upon  • > 
racial  identity.  _ i « 


Income 


. ■ M-.'-  .X,-"*  . ■«;si '•nsi'i.Tr 


Income  is  the  most  quantifiable  aspect  of  welfare. 
Albeit  obvious  that  American  minority  group  menbers  as  a 
whole  are  better  off  than  the  majority  of  the  populations 
of  most  underdeveloped  countries,  they  are  much  worse  off 
than  their  white  fellow  citizens  as  a whole,  in  terms  of 
the  indicators  of  underdevelopment  discussed  above,  the 
following  examples  point  out  the  disparities  between  this 
sector  and  the  larger  society.  The  percentage  of  minority 
group  members  below  the  poverty  line  was  approximately  t%#o 
and  one-half  tiroes  the  percentage  of  the  entire  society 
belw  the  poverty  line  for  each  year  between  1959  and  1971.® 
Median  family  income  for  minority  group  members  as  a per- 
®®**tage  of  white  median  family  income  has  steadily  increased 
from  51  percent  in  1947  to  63  percent  in  1971,  however  the 
dollar  difference  in  median  family  income  has  also  steadily 
increased . 


Resource  Ownership 


Many  economists  believe  that  capital  occupies  a central 
position  in  the  theory  of  economic  development.  It  performs 
double  duty  in  that  it  increases  current  aggregate  demand 
through  its  role  as  investment,  while  simultaneously  insuring 
that  the  capeU>ility  of  the  economy  to  produce  in  future 


302 


u 

o 

v_.' 

kJ 

w 

U 

U 
( ■ 


periods  is  increased.  Human  capital  as  wc-ll  as  phvsical 
capital  foraation  is  important  to  economic  development. 
Examples  of  human  capital  formation  include  expenditures 
for  education,  skills  improvement,  health,  and  internal  ^ ■ 
migration  to  allow  the  population  to  take  advantage  of  better 
job  opportunities.  The  minority  group  sector  possesses  less 
human  capital  than  the  average  for  the  society.  ,,  at  ^4^ 

It  is  possible  to  arrive  at  more  precise  estimates  of 
the  lag  in  ownership  of  quantifiable  ohysical  capital. 
Minority  group  members  constituted  17  percent  of  the  United 
States  population  in  1969.  In  that  same  year  they  owned  4.3 
percent  of  all  business  enterprises  in  the  county  and  re- 
ceived only  .7  percent  of  all  business  receipts.  This  lack 
of  profitable  business  ownership  affects  both  the  prospects 
for  upward  socioeconomic  mobility  and  the  current  income 
position  of  minority  group  members.  The  Section  8(a)  Pro-  •' * 
gram  was  an  explicit  attempt  to  improve  the  economic  pro-  i'’ 
epccts  for  this  group  by  improving  its  business  prospects.  1*. 
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Historically  unemployment  rates  for  minority  group 
mei4bcrs  have  been  about  twice  as  great  as  that  for  white 
members  of  the  society.  This  has  been  true  regardless  of 
the  age  or  sex  of  the  group  that  is  compared  or  the  general 
economic  climate  at  the  time  of  the  comparison.  Under- 
employment is  a far  more  pervasive  and  insidious  problem 
for  minority  economic  development  prospects.  It  is  likely 
that  underemployment  is  much  more  prevalent  among  minority 
group  members  than  the  larger  society  owing  to  «-h*>  job 
opportunity  structure. 

RATIONALE  FOR  SECTION  8(a) 
r'  - ^ - program  IMPLEMENTATION  ' / 


The  Section  8 (a)  Progreun  was  bom  out  of  the  civil 
rights  progress  and  civil  strife  of  the  last  two  decades. 

The  guarantee  of  the  full  power  of  citizenship  without  the 
encumberances  of  discrimination  was  implicit  in  the  latrs 
passed  and  executive  orders  issued  on  behalf  of  minority 
group  members.  This  program  was  but  one  part  of  the  Federal 
Government's  Minority  Business  Enterprise  effort.  It  sought 
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to  co^lenent  congressional  action  to  prohibit  discrimination 
^ housing,  employment,  and  educational  opportunity 

by  seeking  to  remedy  a manifestation  of  unequal  access  to 
one  type  of  remweration — business  ownership.  The  National 
Commission  on  Civil  Disorders  concluded  that  the  riots 
w^ch  rocked  the  ghetto  areas  of  major  cities  in  the  siiddle 
1960s  resulted  from  a sense  of  alienation  from  the  social 
and  econo^c  systems  of  the  country  on  the  part  of  minority 
group  members.  The  Commission's  report  stated  that. 


Wie  expectations  aroused  by  the  great  judicial  and 
J^i*^tive  victories  of  the  civil  rights  nmrrmrnt  n have 
**  to  frustration,  hostility,  and  cynicism  in  the  face 
' • of  the  persistent  gap  between  promise  and  fulfillmentr 

\ 

Defusing  the  riot  environment  tras  a major  goal  of 
President  Johnson's  Test  Cities  Program  in  which  the  idea 
for  the  Section  8(a)  Program  made  its  debut.  Thus,  the 
Sect:ioii  8(a)  Progm  was  reactive  in  its  attempt  to  provide 
for  equality  of  oiportunity  and  simultaneously  stabilize  a 
threat  to  the  fabric  of  the  society.  The  program  took  on 
its  current  form  through  a series  of  executive  orders  and 
memoranda  with  congressional  knowledge  and  encouragement. 


Federal  procurement  is  big  business.  Federal  contract 
awards  amounted  to  more  than  $52  billion  in  fiscal  year  1974. 
Congr^s  has  declared  its  intent  to  monitor  and  influence 
the  distribution  of  benefits  from  these  expenditures  since 
it  passed  the  Armed  Services  Procurement  Act  in  1947. 


ECOHONIC  DEVELOPMENT  EFFECTIVENESS 
AND  COST  OP  THE  SECTION  8 (a)  PROGRAM 


The  quantitative  portion  of  this  study  was  comprised 
of  an  analysis  of  the  economic  development  effectiveness  and 
cost  of  Air  Force  participation  in  the  Section  8(a)  Program. 


Development  Effectiveness  of  Air  Force  Participation 

^ This  ^ research  used  a multiple  linear  regression  analysis 
to  investigate  the  economic  development  effectiveness  of 
Air  Force  participation  in  the  Section  8(a)  Program.  The 
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analysis  was  conducted  on  12  variables  associated  with 
935  Air  Force  Section  8(a)  ctMitracts  awarded  to  365  firms 
between  1968  and  1976.  Two  separate  regression  analyses 
were  conducted  on  the  effectiveness  of  Air  Force  partici- 
pation in  the  program.  The  regression  analyses  postulated 
a linear  relationship  betvieen  the  variables  am^  was  expressed 
ast  -•  ^ . 

‘ “ ' ' t • ■’  '>4 

Development  '•  ’ nA  ' • 

Effectiveness^  S - f(N,  Y,  T,  D,G,  M,  V,  P,  C,  R)  (Equation  1) 

Nherei  ' ' ’'f/  viV-’ r 

’ ■ ’ ■ jrHJoxsl'ivpil' 

S  ■ Total  dollar  amount  of  Air  Force  Section  8(a)  contracts  ’ 
awarded  this  firm  since  FY  1968 

M » Number  of  Air  Force  Section  8(a)  contracts  awarded  the 
firm 

Y * Fiscal  year  in  which  the  first  Section  8(a)  contract  wa&; 
awarded  the  firm  . ' 


r ' , i ! itfi' r iTv  iiA 

T ■ Type  of  contracting  done  by  the  firm:  ^ •* 

1 - M2uiufacturing  i . ^ . 2 -'oV 

2 ■ Service 

3 » Construction  . , ■ , 

< «.  j, 

D > Percentage  of  the  population  idiich  was  nonwhite  in  the 
city  of  contract  award 

. ■ - A 

G ■ Minority  group  of  the  firm:  * i ^ 

1 ■ Black  ' 

2 ■ Puerto  Rican  ■ t -n  ■;« 

3 « Americ^m  Indian 

4 ■ Spanish  American  . ' -s-  ‘4^=.:..  a 

5 « Asian  . ‘ 

6 * Eskimos  and  Aluets  . 

7 « Undetermined  (none  in  the  sample  analyzed) 

8 « Others  - including  whites  ^ 

I • 

M » Total  number  of  Section  8(a)  contracts  awarded  this 

firm  by  all  other  federal  agencies  ..  . 


V Total  dollar  value  of  Section  8(a)  contracts  awarded 
this  firm  by  all  other  federal  agencies 

P « Population  size  of  the  city  of  award  in  1970 


C • Business  class  of  the  firm: 

1 ■ Manufacturing 

2 ■ Construction 

3 « Service 
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' . iJ-ve  ilk 

ocf,J 

R - Snail  Business  Administration  region  in  which  the  firm  ' ^ 
was  located  (regions  are  numbered  1 through  10) 

An  alternative  formulation  of  the  nodel  specifies  the  *.  r .- 
f variable  as  the  dollar  amount  of  each  Air  Force  • . - 

Section  8 (a)  contract  written  and  was  expressed  ass 

Development  ' ^ 

Effectiveness  A - f (S,N,Y,T,D,G,M.V,P.C,R)  (Equation  2)  u . 

V .W'AXi  dfrAiij  , 

A - Dollar  aootml:  o»  ' 


(ol 


Hheret 

V’  ‘ *r 

Dollar  amount  of  this  contract 
Year  of  contract  award 


. «•  :i  r.vtjv;  ni  ias£ 

'Tt  I ^ r*  " ■ 3 ^ t-'A/h 

All  other  variables  are  as  specified  in  equation  1. 

_ , . _ ' ' • ■ ' - . . y T'  «■  V 

The  linear  form  of  the  equations  was  expressed  as 
follows:  £ 

f*  ^ ^ j.  r 

^ * Alphiis  ♦ 1>|  N ♦ ^ b«%T  y ^ t>  P 

bjC  ♦ u,  andt  ^ ^ i ^ .5_  (E|ILitlS  la)  ® 

• \ f “ . 

A ■ Alpha  ♦ beS  -f  b^N  + b-^Y  ‘fbT4-bD'fbG4'  hJi  4*  h v 4> 
b,P  ♦ a bjjH  * „ 2 3 < . 5 (6,„atiS  2.y 

Where*  ^ * 

j 

Alpha  is  the  intercept  term,  b,  through  b, , are  the  beta 
cr. efficients,  and  u is  a randoAly  distribQted  error  tern 
with  mean  zeroe  . ,, 

‘1 

The  data  for  the  research  was  extracted  from  the  Small 
Business  Administration's  Status  Report  of  8(a)  Contracts 
and  the  Department  of  Commerce,  Bureau  of  the  Census'  1973 
and  City  Data  Book  and  1970  Census  of  Population • 

General  Social  and  Economic  Characteristics  PCllT  series'. 

Thirty  two  Air  Force  contracts  awarded  to  19  firms  during 
fiscal  year  1968—1976  time  period  were  excluded  from  the 
analysis  because  all  relevant  data  on  them  could  not  be 
obtained. 
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The  regression  results  indicated  that  the  Section  8(a) 
Program  was  a positive  economic  development  tool.  The  P-test 
ratio  was  statistically  significant  at  the  99  percent  con- 
fidrace  level  for  both  specifications  of  the  independent 
variable.  The  F-test  ratio  was  5.10  for  equation  1 and 
15.51  for  equation  2.  The  critical  numbers  for  sauries  of 
these  sizes  and  numbers  of  degrees  of  freedom  were  2.70  for 
equation  1 and  2.66  for  equation  2.  Table  1 summarizss  the 
statistical  resultsil^ndivldual  independent  variables. 


Cost  of  Air  Fbrce  yairticlpation 

The  research  attempted  to  contruct  a rough  cost 
estimate  for  Air  Force  participation  in  the  Section  8(a) 
Program.  The  vehicle  for  the  cost  estimate  was  a survey 
questionnaire  trhich  was  sent  to  81  active  Air  Force 
installation  procurement  offices.  The  estimate  was  based 
on  answers  contained  in  the  74  questionnaires  which  %fere 
returned  (t%«o  questionnaires  were  returned  later  which  did 
not  alter  the  results) . The  formula  csed  for  computing  dollar 
costs  extracted  the  modal  grade  of  the  procurement  offices* 
Small  Business  Specialist  and  multiplied  the  midpoint  of  the 
salary  for  that  grade  by  the  midpoint  of  the  modal  class  of 
time  spent  on  the  program.  It  added  an  increiOnent  of  salary 
for  assistants  to  the  Small  Business  Specialists^  computed 
in  a like  manner.  A summary  of  the  computation  for  the  cost 
of  the  program  to  81  active  Air  Force . installation  is  as 


c 

follows: 

( 

Salary  paid  to  Small  Business 
Specialists 

$21,970 

Percent  of  time  spent  on  Section 
8(a)  procurement 

2.5 

Salary  paid  to  other  procurement 
personnel 

$17,523 

: 

\ 

Percent  of  time  spent  on  Section 

8 (a)  procurement 

2.5 

Estimated  dollar  cost  of  time  spent  per  installation  per  year 
($21,979  X .025)  + ($17,523  x .025)  « $549.25  + $438.08  « 
$987.33.  The  agregate  dollar  cost  to  the  Air  Force  for  parti- 
cipation is  then:  $987.33  x 81  » $79,973.73. 
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The  respondents  to  the  questionaire  were  asked  to 
estimate  the  nonraonetary  cost  of  the  program  to  the  Air 
Force  through  a series  of  questions  about  the  impact  of 
the  program.  When  they  were  asked  to  rate  the  program  on 
a scale  of  1 to  9 in  terms  of  excessive  cost  to  the  pro- 
curement  process  compared  to  normal  procurement,  the 
average  response  was  6.47.  When  asked  whether  the  program 
had  adverse  li^>acts  on  the  missions  of  their  organizations 
and  the  mission  of  the  Air  Force,  72  and  74  percent,  re- 
spectively of  the  respondents  thought  it  did  not.  Fifty 
seven  percent  of  the  respondents  thought  that  the  program 
had  more  favorable  than  unfavorable  impacts. 


Federal  public  policj  activities  are  deemed  to  be 
warranted  when  the  externalities  usoclated  with  some 
phenomena  can  only  be  captured  at  the  national  level. 

Minority  group  economic  underdevelopment  is  such  a 
I^ienomenon.  The  analysis  associated  with  this  research 
effort  suggests  that  the  Section  8(a)  Program  is  a useful 
tool  with  iriiich  to  combat  the  business  underdevelopment 
aspect  of  the  minority  group  economic  underdevelopwnt 
problem.  The  statistical  results  indicate  that  increased 
emphasis  on  the  progrem  should  yield  increased  development 
effectiveness.  The  program  should  not  be  expected  to  do 
more  than  provide  an  equal  opportunity  for  minority  group 
businessmen  to  succeed  or  fail  on  their  merits.  It  appears 
to  be  meeting  this  challenge. 

The  program  was  not  found  to  be  inordinately  expensive 
to  the  procurement  process  either  in  terms  of  dollar  costs, 
excessive  procurement  cost,  or  adverse  mission  impacts.  It 
is  likely  that  the  same  is  true  in  other  federal  agencies. 

More  research  is  needed  to  facilitate  evaluation  of 
the  cost  and  effectiveness  of  socioeconomic  programs  in 
accomplishing  their  stated  goals.  Just  as  budget  constraints 
in  the  Department  of  Defense  do  not  permit  the  dedication 
of  funds  to  ill-conceived  or  ill-planned  programs,  national 
resource  allocation  considerations  dictate  that  benefits 
ere  reaped  coimensurate  with  money  expended  in  pursuit  of 
national  goals. 
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^ “ practicing  electronic  engineer  within  the  Departwnt  of 
Mfense*  I have  often  wondered  »rhy  I ever  bothered  to  earn  a degree 
In  engineering.  All  of  those  long  nights  studying  advanced  calculus, 
tran^sslon  lines*  and  AC  Machinery  was  for  naught.  This  fact  becoaws 
evident  when  one  graduates  Into  the  engineering  nanageaent 
field.  Now  your  need  for  engineering  skills  declines  expotentlally 
and  your  need  for  business  and  accounting  knowledge  increases  geo- 

"street  wariness*'  Is  now  required.  The 
old  skills  that  have  for  so  many  years  carried  you  up  the  ladder  are 
useless.  No  longer  does  two  plus  two  always  equal  four.  The  fact 
that  yw  are  or  were  a good  engineer  becomes  absolutely  meaningless, 
suddenly,  after  so  many  years  of  warmth  and  security  the  unsuspecting 
engineer  Is  thrown  out  Into  a dark  cold  world.  Scared  and  very  dis- 
oriented, he  Is  ready  prey  for  battle  scarred  contracting  officers 
nervous  accountants  and  havkeyed  Barketeers* 


^ f®’'  months  pass  and  the  fledgling  engineer  manager  has  had 
time  to  test  his  wings.  He  has  discovered  that  In  management  and 
contracting  everyone  does  not  tell  the  whole  truth  (on  the  other  hand, 
engineers  never  lie  to  one  another).  He  has  discovered  that  reports, 
unagement  information  systems,  and  organizational  charts  are  the 
least  of  his  worries.  He  is  now  the  leader  of  a group  of  ten  or  so 
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engineers  and  technicians  who  all  want  to  go  their  sei^rate  ways. 

Alternately,  the  new  manager  must  be  chaplain,  father,  sheriff,  and 

sea  lawyer.  Thinking  that  all  that  can  happen  to  him  has  happened,  * 

he  Is  suddenly  confronted  by  the  Coimanding  Officer  of  his  activity 

and  asked  to  supervise  work  on  a new  contract  effort.  . ^ 

Perhaps  a little  background  Information  Is  necessary  to  dis- 
cover how  our  naive  engineer  became  entangled  In  this  trap.  ' 

1 1975  the  NAVELEXSTSENGCEN  Portsoiouth  took  occupancy  of 

a rented  28,000  square  foot  former  appliance  center  located  at  7454 
North  Military  Highway,  Norfolk,  Virginia.  NAVELEX  Headquarters, 

Washington  had  given  the  go  ahead  to  establish  an  east  coast  electronic 
repair  facility  In  the  Norfolk  area.  The  original  plan  called  for 
a build  up  over  a three  year  period  to  150  civil  service  electronics 
workers.  The  facility  was  named  EMC,  the  Electronic  Maintenance  Center. 

Five  months  to  the  day  after  the  building  was  rented,  the  doors  ** 

were  open  for  actual  production  work.  However,  as  NAVELEX  Portsmouth 
started  to  implement  the  original  manning  plan  it  was  discovered  that 
temporary  constraints  precluded  celllsg  points  being  made  available 

for  the  new  Electronics  Maintenance  Center.  In  order  to  operate,  • — ' 

the  facility  was  staffed  with  temporary  Civil  Service  employees.*  It 

was  hoped  that  these  temporary  positions  could  be  covered  to  celling  I 

points  upon  their  expiration  date  of  1 July  1976.  To  the  dismay  of 

NAVELEX  Portsmouth,  MAVELEXSYSCOM  Washington  notified  the  command 

that  ceUlng  points  would  not  be  made  available  and  that  other  staffing 

arrangements  would  have  to  be  made  for  continued  operation.  . 

In  April  1976,  our  engineering  friend  was  asked  to  assume  temporary  ' 

unagement  responsibility  for  the  EMC  facility.  This  f-as  necessitated  I 

by  the  retirement  of  the  supervisor  responsible  for  the  QIC  operation.  ! 

This  temporary  assignment  was  to  be  In  addition  to  his  present  position  t 

as  Head  of  the  Test  Measurement  and  Diagnostic  Equipment  and  Radiac  ! 

Division  of  NAVELEX  Portsmouth.  Little  did  he  know  at  this  time  what  i • 

lay.  ahead.  . ‘ 

The  future  of  the  Electronic  Maintenance  Center  was  very  mich  in  | 

doubt  during  the  spring  and  early  summer  of  1976.  The  only  way  to 
continue  operation  of  the  facility  after  July  1976  would  be  some  type 
of  contracting  effort.  At  this  time,  the  EMC  facility  employed  some 
fifty  temporary  and  six  permanent  civil  service  employees.  Due  to  the 
short  history  of  the  maintenance  center,  adequate  specifications  were 
not  available  to  solicit  competition,  and  time  did  not  allow  for  sub- 
■ission  of  technical  proposals. 

Now  what?  Time  was  short.  The  contracting  effort  for  the  EMC 
operation  was  a somewhat  difficult  task  even  for  an  experienced  person, 
much  less  a relatively  inexperienced  engineer. 
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^ often  happens  when  things  appear  to  be  hopeless,  an  unexpected 
ray  of  light  shown  thru  the  clouds. 

-Superior  Engineering  and  Electronics  Company  of  Los  Angeles 
California  learned  of  the  situation  and  submitted  an  unsolicited* 
pro^sal  for  the  operation  of  the  Norfolk  facility  to  NAVELEXSYSCOM 
Washington.  This  proposal  was  forwarded  to  NAVELEX  Portsmouth  for 
consideration.  Superior  is  a one-hundred  percent  minority  owned 
company  certified  eligible  under  the  8(a)  program.  Their  area  of 
speclalleatlon  Includes  shipboard  systems  integration,  engineering, 
overhaul,  fabrication.  Installation,  modification,  and  checkout  of 
electronic  and  ordnance  systems.  In  November  of  1975  they  were 
selected  as  one  of  the  outstanding  small  business  firms  In  tbelr 
region  (region  nine  (9)). 

Everyone  at  NESEC  Ports  was  somewhat  skeptical.  It  was  a ' 
foregone  conclusion  that  the  8(a)  program  was  a training  program, 
tecause  there  is  no  competitive  bidding  and  owing  to  the  fact  most 
8(a)  contrTCts  contain  what  amounts  to  learning  process  subsidies, 
the  cost  of  the  contract  would  necessarily  have  to  be  great 
The  decision  came  down  to  this;  was  It  better  to  try  an  8(a)  con- 

and  do  no  repair  work  at  all.  Subse4;uently, 
Washington,  In  accordance  with  DOD  policy  (ASPR 
i Indicated  a desire  to  award  an  8(a)  contract  to  a highly 
qualified  firm  for  the  operation  of  the  Norfolk  facility. 


The  decision  was  made  and  work  was  started  on  the  contract. 
During  the  period  Kay  to  September  1976,  many  hours  were  spent  in 
*^^^^**8  requirements  and  deciding  on  what  functions  would  be  con- 
tracted and  which  would  reoain  the  responsibility  of  the  government. 
In  June  of  that  year,  a rough  solicitation  was  completed.  Working 
very  closely  with  the  Negotiator  and  Contracting  Officer  from  the 
Regional  Procurement  Department  of  the  Naval  Supply  Center  Norfolk 
NKEC  Portsmouth  refined  the  scope  of  the  proposed  contract  and 
closed  In  on  what  it  considered  to  be  a final  solicitation. 


The  original  solicitation  called  for  a combined  firm  fixed 
price  and  tine  and  materials,  indefinite  delivery  type  contract. 

This  t^e  contract  calls  for  submission  of  clearance  prior  to  place- 
ment of  individual  orders.  Negotiations  with  Superior  began  in  July. 
A reprise  was  granted  to  extend  all  temporary  employees  through  the 

this,  uncertainty  and  attrition 

had  reduced  the  work  force  at  EMC  to  thirty  temporary  Civil  Service 
employees . 


During  negotiations.  It  was  determined  that  a chauge  in  contract 
type  was  in  order.  It  was  decided  that  a cost  plus  award  fee  contract 
would  be  to  the  mutual  advantage  of  the  Government  and  Contractor. 

On  a closer  examination,  it  was  discovered  that  a further  breakdown 
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of  services  would  be  required;  the  manageaent  of  the  facility,  and 
the  direct  labor  effort.  On  advice  froa  the  NSC  Norfolk  Procurement 
Department,  the  management  portion  wuld  be  a firm  fixed  price  for  a 
one  year  period  based  on  a labor  effort  of  75  (+  25Z)  direct  production 
personnel.  The  direct  labor  effort  would  be  a cost  plus  award  fee 
because  it  %rould  be  impossible  at  the  time  of  contract  award  to  know 
with  any  degree  of  certainty  the  level  of  work  or  amount  of  materials 
necessary  to  perform  the  required  overhaul.  Both  the  government  and 
contractor  %iould  derive  certain  benefits  from  the  combination  of  firm 
fixed  price  and  cost  plus  award  fee. 

By  mid  July,  final  negotiations  were  in  progress.  A meeting 
was  held  with  representatives  of  Superior  Engineering,  NSC  Norfolk 
and  NAVELEX  Portsmouth  in  attendance.  A final  contract  agreesmnt 
was  reached.  The  NSC  Procurement  Department  wrote  the  business 
clearance  and  forwarded  it  to  Chief  of  Naval  Materials  for  approval. 
Time  ms  growing  very  short.  In  late  August,  approval  was  finally 
received  from  NAVMAT.  One  last  hurdle  remained,  the  contract 
to  be  awarded  by  The  Small  Business  Administration.  In  mid  September, 
the  contract  was  finally  awarded.  Contract N00189-76-C-0644  with 
Superior  Engineering  and  Electronics  Co.,  Inc.  was  a fact  of  life.  The 
contract  from  start  to  finish  had  been  completed  in  seventy  eight  work- 
ing days.  Ten  days  short  of  the  "drop-dead**  date  of  1 October  1976. 


CONINACT  TYPE 

The  required  services  have  been  broken  down  into  two  areas,  i.e., 
management  of  the  facility  and  direct  labor  effort  (Item  OOOIAB). 

The  management  portion  shall  be  a firm  fixed  price  for  a period 
of  1 year  based  on  a labor  effort  of  75  (+  25Z)  direct  production 
personnel.  The  direct  labor  effort  will  be  on  a cost  plus  award  fee  basis 
because  it  would  not  be  possible  at  the  time  of  contract  award  to  know 
with  any  degree  of  certainty  the  level  of  work  or  amount  of  materials 
necessary  to  perform  the  required  overhauls. 

Both  the  Government  and  contractor  will  derive  certain  benefits 
from  the  combination  of  firm  fixed  price  and  cost  plus  award  fee.  Since 
the  Government  is  providing  the  facilities,  utilities,  and  special  test 
equipment  the  contractor  can  estimate  with  a high  degree  of  certainty 
the  management  effort  needed  to  operate  the  facility.  By  considering 
the  management  portion  on  a firm  fixed  price  basis  a substantial  amount 
of  what  usually  would  be  considered  overhead  has  been  deleted.  The 
contractor  has  incentive  to  control  his  management  expenditures  wisely 
and  the  Government  saves  on  administrative  effort  by  not  having  to 
verify  management  costs.  The  contractor  is  assured  of  recovering 
all  allowable  costs  on  his  direct  labor  and  will  be  awarded  for  accurate, 
timely,  cost  effective  work.  , 
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/ 


unit  J^^r**^**  ‘**®®‘*  ®“  P«fcent  of  work  units  (esch 

to  be  repaired)  conpleted  on  time,  the  percent  of  work  untro 

wt:j:ih:”es“i::t^“'‘L'^;‘’^’  orcirk^isL"  ziite^ 

thin  the  estimate.  Each  factor  will  be  weighted  as  follows: 


Tlaeliness  ->  50Z 


* 25Z  Cost  Accuracy  • 25Z 


8S  locentlve  effectiveness  has  been  established  to  be 

Tt  JL^Ia^rntT*^  “ experience  of  Government  i 

Evaluation  will  be  performed  on  a quarterly  basis  as  follows: 

«^ts  completed  on  r<,«  , 50  - (j)  weighted  percent 
I»Mber  of  work  units  issued  ! . percent 


2. 

3. 


of  iwrk  mnlts 


f I?  ^w-ctto.  .25-0)  ■ 

percent 

««nlts  completed  at  or  below  estimate  x 25  - (3)  weishted 
■■bar  of  work  units  issu^  ^ weighted 

. percent 


tj  V , 


Weighted  Evaluation  Percent  fif  greater  than  6ST  x Total  Award  Fee 

85  I 

**f***^*^  evaluation  percent  is  less  than  65  percent  award  eerft-t^ 

r^o^“id“r:^  *"**'*•■ 

by  the  contractor  a!  to  dellve^  datrand^oI^^^Th^Giv^^JJ^JJ^ 

«rl!ld  ^ changes  must  be  agreed  to  by  the  Government.  The 

assignment  or  modified  work  assignment  will  be  the 
basis  for  the  applied  award  for  evaluation. 

Cost  Breakdown  and  Negotiation  Results 

rKi-  ^ Superior  has  had  a contract  of 

ype,  actual  or  estimated,  cost  factors  in  previous  contracts 
o not  exist.  There  are  no  known  recent  operating  statements  of  the 
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proposed  contractor  for  this  type  of  work.  An  audit  was  conducted 
by  DCAA  Los  Angeles.  All  contractor  proposed  rates  were  approved; 
however  $30»000  of  other  direct  costs  were  unsupported  ($25,000 
miscellaneous  -b  5,000  for  a sign).  Wages  proposed  are  the  current 
rates  and  are  considered  fair  and  reasonable. 

The  contract  Is  broken  down  into  two  categories,  l.e.»  admin- 
istrative and  direct  production  Labor.  Each  category  will  be  con- 
sidered separate,  and  individual  objectives  were  established  by  the 
procuring  activity.  The  following  breakdown  is  a suassary  and  backup 
data  for  contractor's  proposal  and  the  negotiated  agreement  on  the 
administrative  portion.  It  was  arrived  at  through  consideration  of 
the  San  Diego  plan  and  the  best  estimate  of  the  current  Government 
management  as  to  what  capabilities  would  be  required  and  what  wages 
should  be  paid  for  the  administrative  personnel. 

The  aggregate  aanageaent  is  composed  of  the  following  subcategories: 
Norfolk  Manageaent  Personnel,  California  Hanageaent  Personnel,  Start 
up.  Training,  Travel,  and  other  direct  costs.  ^ 


I.  Management  Effort  (FFP) 


A. 

Norfolk  Managcm.int  Personnel 

189,257 

B. 

California  Management 

Personnel 

84,487 

C. 

Start-up 

51,313 

D. 

Training 

. 

33,570 

E. 

Travel 

7,320 

F. 

Other  Direct  Cbsts 

42,300 

Sub-total 

408,247 

G. 

G6A  914. 95Z 

61.033 

Sub-total 

469,280 

H. 

Profit 

(lOX) 

46,928 

I.  Consents  on  Direct  Overhead 


516,208 


(81) 


Negotiated 

189,257 

72,046 

15,055 

18,951 

7,320 

30.800 

333,429 

49,848 

383,276 
30,662 


413,939 


L 


II.  Direct  Production  Effort  CPAF 

This  is  the  CPAF  portion  of  the  contract  and  provides  for  the 
direct  labor  production  effort.  The  categories  of  labor  to  be  used  are 
as  follows:  Foreman,  Electronic  Mechanic,  Electronic  Worker,  Electronic 
Helper,  and  Electronic  Assistant.  Based  on  an  estimate  of  seventy— five 
man-years,  a level  of  effort  of  162,067  man-hours,  including  overtime, 
has  been  established.  Position  descriptions  are  included  as  part 
of  the  contract. 

It  is  estiuated  that  material  and  subcontracting  mrk  of 
approximately  $400,000  will  be  required  during  the  contract  period. 

The  estimate  of  total  cost  for  75  (plus/minus  25Z)  man-years  of 
direct  production  effort  is  as  follows: 


317 


1 


Direct  Labor 
Overhead  0 25. 8Z 

• .TO 

Material 


v:.; 


$779,024 

200,988 

11,092 

400.000 


SUBTOTAL 
Cli  A 8 14.95X 


s*  I t 

•- VI  > ' 


TOTAL  DIBECT  COST 
Baae  Fee  8 2Z 
Award  Fee  8 4Z  ^ 


S>  nc  _ 

...  Total  Cost 


1,391,104 

. 1.599,074 
31,981 
63.963 


t 11 


$1,695,018 
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tti<  «r**  El^tronlc  Maintenance  Center  currently  charges  Its  custoiners 
^1  * ^1.****^  direct  labor  hour.  This  charge,  exclusive  of  aaterial,  is 
also  the  current  charge  by  Haval  Shipyards  doing  this  type  of  work. 

- - • 

».u  to  calculate  cost  per  hour  (exclusive  of  material)  to  ^ 

hllle^nr^d^“^**^  contract,  the  following  calculations 

613,939  • 

1.695.018  • • ' 

400.000  • ••• 


1^: 


Managenent  Cost 

Total  Est.  Cost  Tins  Fee 


Delete  Material 
TOTAL  COST 


1,708,957 


h....  I®  saterial,  the  Covernnent’s  over- 

llnt  e*ii“ted  that  overhead  of  $220,000  covering 

rent,  utilities,  Covemsent  personnel,  etc.  is  an  accurate  figure. 


TOTAL  COST 
COVERMMENT  OVERHEAD 

TOTAL 


1.708.957 
220.000 

1.928.957 


the  level  of  effort  is  142.067  man-hours  the  dollar  per 
man-hour  is  calculated  as  follows:  1,928,957  i 142,067  - $13.58 

.K  4 in  the  local  area  performing  this  type  of  work  in 

wirklr^t^liill^  approximately  150  direct  production 

workers)  typically  charge  between  12.00  and  13.00  per  direct  labor  hour. 

M..  ““^ement  portion  is  fixed,  as  is  the  maximum  fee, 

the  $/hr.  figure  decreases  as  the  level  of  effort  increases.  If  the 
indei  reaches  93  direct  production  personnel  (the  maximum 

be  $12  SO*  * ®®“^  of  material,  will 

be  $12.50.  These  figures  compare  favorably  to  any  open  market  situation 
and  are  substantial  savings  to  the  Government. 
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The  basic  concept  driving  the  EMC  operation  is  that  price  and 
mrknanship  Bust  be  of  paraaount  iaportance.  Each  Job  received  at 
the  facility  must  be  priced  out  coapletely  and  approved  by  the 
govemnent  prior  to  start  of  work.  The  basic  flow  of  work  thm 
the  facility  is  as  follows: 


New 


EMC 

vProductioQ 


. i 

_y^ntractor 


l|L«ncraccor___vqA  _ 
(Superior)  (govt) 


Vork  'Control 


.lioW  Cv  .>ff7  , .JiaQ4' avaj 

NAVELEX  Ports  will  have  cooplete  control  over  the  contractor *8 
kloadg  schedule  and  quality  assurance.  ^ " ; 


, « One  of  the  first  steps  that  was  taken  after  the  contract  award 
was  to  size  up  the  competition.  The  following  information  was  gathered 
and  a copy  was  posted  on  all  bulletin  boards. 

• *•-  '*  . *1  > 'SJfirWl 

' . ActlTltT  • 


a*'  1 . ^ ^ » I , ^ 

,^,1  HESEC  San  Olego.  CA 


t'a)-?  sr.  atftrt  ?.-i  ^ 

^ $16.00 


‘f  i- 


NNSY  Portsmouth^  VA  " . 

NNST  Portsmouth,  Mew  Hampshire 
NST  Charleston,  SC. 

OTHER  LOCAL  CONTRACTORS 


’a 

^ 9 


i J > 


$16.00 
$20.00 
$20.00 
$20.00 
$12.00-$20.00 


»•??  ax  yi 
'-ii  * vj 

r, 

' **  r.Ki 
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It  was  absolutely  necessary  for  the  EMC  Facility  to  keep  its 
price  below  $16.00  per  man  hour. 


expansion 

plan  worked 

out  as  follows: 

SUPERIOR 

SUPERIOR 

TOTAL 

MANNING 

OVERHEAD 

GOVT 

LABOR 

COST 

35 

$6.85 

$3.42 

$6.79-$9.46 

$17.00-$19.73 

55 

4.36 

2.17 

$6.79-$9.46 

$13.32-$15.99 

100 

3.12 

1.19 

$6.79-$9.46 

$11.10-$13.77 

150 

2.49 

.80 

$6.79-$9.46 

$10.08-$12.75 

From  this  chart,  it  was  evident  that  as  the  workload  increased 
and  the  fixed  cost  were  spread  over  additional  manhours,  the  total 
cost  per  manhour  took  a drastic  drop.  Using  this  as  a selling 
point,  EMC  proceeded  to  expand  its  work  and  subsequently  decrease 
its  cost. 
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*’  "“*■  facility  revealed  that  a work 

force  of  100  people  could  be  easily  boused.  Figure  (1)  was  used 

capIblllJj.*  co«uKls  our  expcr^lon 

■ arr-y^.  - ■y.rf  . ■ ■ • ■ ...  •<-  >.j3F- 

^ ^ . 

8(a)  contract  Ir  in  ita  eighty  sooth.  To  date  the  rMuir. 

«x W;.  £1 

S'S-L?'  s- - - - £S?- 


Many  special  projects  have  been  soccessfullv  a ..u 

quality  of  the  work  produced  has  re^^HS  cospleted  «,d  the 


r 

It  ^ interesting  to  note  that  an  8(a)  fit*  has  been  .hi.  »« 
CM^te  head-on  with  other  govemnent  activities  and  private  flrw 
fron  a cost  standpoint.  When  the  contract  first  started  RESEC 

“*  t"*"* 

h!!iiiJ  “PPly  center  requested  for  stock.  Gradually,  we  have 

built  this  up  to  where  we  now  do  50  percent  of  these^^lJ!! 


'***  Facility  as  a viable  and  cost  effect 

fiSlI^  ^ *«<»«  at  tJe 

facility  looks  like  it  will  double  in  5T78. 


Havy: 


This  contract  effort  solved  three  najor  probli 


areas  for  the 


m/  \ 

(a)  It  elininated  the  need  for  additional  ceiling  points 


(b)  It  placed  work  in  the  snail  business  sector,  nore 
especially  in  the  8(a)  sector 


(c)  It  provided  the  Navy  with  a urgently  needed  product 
qu^lty*^  * reasonable  cost  and  of  the  required 


Ftot  all  points  of  view,  this  first  eight  months  have  been 
successful  ones.  The  goal  of  NESEC  Portsmouth  is  to  assist  Superior 
every  possible  way  so  that  the  company  can  "graduate"  from  the 

^ compete  in  the  business  world  without 

thl^^jilSjy*™*”'  assistance).  Superior  is  rapidly  approaching 
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Lessons  Learned 


With  any  contract » 8(a)  or  othervise*  there  are  adjustiients 
and  problea  areas  that  aust  be  worked  out  after  the  fact.  The  tlae 
is  rapidly  approaching  when  a new  contract  for  FY78  oust  be 


negotiated  and  let.  . - 

Froa  past  experience,  HESEC  Ports  intends  to  aodify  the  upcoising 
contract  so  that  additional  responsibility  is  placed  on  the  contractors. 

U Enough  pricing  inforaation  is  now  available  to  shift  aany  of  oiir  repair 
efforts  to  a Fira  Fixed  Price  contract.  It  has  been  suggested  that 
new  lines  of  work  coaing  into  the  facility  that  lack  adequate  price 

U inforaation  could  be  placed  on  an  indefinite  quality-tiae  and  aaterial 
contract  with  Superior.  ^ 

UThis  first  year  contract  is  being  used  as  a refincaent  stage  to  ^ 
iaprove  prices,  services,  and  response  tiae  in  succeeding  contracts. 

The  FT78  contract  will  be  a big  iaproveaent  over  this  contract,  and 
the  price  and  product  will  be  further  iaproved.  ..  ,, 


A Value  Engineering  clause  will  be  added  to  the  new  contract. 
Awards/award  fees  granted  under  tUs  prograa  sbould  be  distributed 
in  a aanner  which  will  proaote  participation  by  all  contractor 
eaployees.  If  a aan  is  held  personally  responsible  for  his  work 
and  knows  that  if  he  does  an  outstanding  Job  he  will  be  rewarded, 
his  production  and  workaanship  will  iaprove.  Delivery  of  a quality 
product  in  a tiaely  aanner  is  a basic  requireaent  for  any  contract. 
Failure  to  correct  those  probleas  which  result  in  substandard  products 
or  failure  to  aaintain  control  of  production  probleas  should  be 
cause  to  consider  contract  teraination  — not  Just  the  loss  of  a few 
dollars  in  award  fee. 
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CONCLUSIONS: 
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The  latest  figures  show  that  about  9.42  million  businesses  in 
the  nation  are  classed  as  small*  They  account  for  43  percent  of 
the  entire  gross  national  product.  They  employ  100  million  Americans. 
Prom  this  point  of  view,  there  is  nothing  small  about  our  small 

businesses.  * % - i 

. . "If.  • *>n'*  c.y.n  rf^ 

The  8(a)  program  has  received  some  bad  publicity.  In  some 
instances,  these  complaints  are  Justified.  The  original  aim  of  the 
8(a)  prc^gram  was  to  aid  businesses  run  by  the  economically  and 
socially  disadvantaged.  This  principle  is  still  a good  one. 


This  does  not  automatically  mean  that  the  government  should  pay 
a higher  price  for  less  goods  or  services.  To  allow  an  8(a)  contractor 
to  deliver  less  in  quality  or  quantity  than  specified  in  the  terms 
of  the  contract  damages  the  small  business  firm  as  %fell  as  the 
government.  _ . . 


We  all  agree  that  there  is  a compelling  national  interest  in 
assisting  minority  owned  companies  to  break  into  the  business  world. 
As  they  prosper  and  become  competitive,  they  in  turn  can  provide 
employment  opportunities  for  others.  In  the  long  run,  this  will 
lower  unemployment  and  conserve  the  tax  dollar. 


In  this  particular  contract,  the  contractor  was  given  no  special 
treatment  that  numerous  other  large  corporations. have  not  been  given 
by  the  government  in  the  past.  They  were  made  to  bid  on  work  rather 
than  providing  a direct  handout.  At  all  times,  the  quality  of  the 
finished  product  was  certified  by  the  government.  Each  and  every 
phase  of  the  contract  effort  was  monitored  and  when  price  or  work- 
manship took  a lapse,  corrective  action  was  taken. 


C) 


Superior  Engineering  and  Electronics  is  a stronger  company  ) 

because  they  were  made  tc  compete  in  a real  life  environment  and 
not  in  the  artificial  world  of  government  subsidies.  This  contract 
has  proven  to  me  that  8(a)  contractors  can  and  will  produce  a quality  , j 

product.  If  the  DOD  is  really  serious  about  providing  assistance  * 

to  minority  companies,  it  will  force  8(a)  contractors  to  be  responsible 
for  their  corporate  actions,  while  to  the  maximum  extent  possible,  / \ 

provide  help  and  guidance  when  needed.  ^ J 


This  8(a)  contract  is  a success! 
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- ' '1  > * EXPAMSION  CAPABILITY  * 

;v. . 

- Personnel:  ■•  ' - 

- •"  r ^ ‘ Work  Benches  Total  7..  V"’  ' . 


• ‘ 1 . 108  Work  Benches  Totsl 
‘^• 

, - ^ \ 32  equipped  for  SRC-20/21  work 

,:  ' * 48  equipped  for  Electronic  Warfare  *iork 

28  equipped  for  22  work 

'-k.*  ■ 

Space  available  for  50  additional  benches 
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* Cleaaing  area 


encnes  . ^ .vcsfaSS 


* Miniature  Repair  capability  with  plans  to  add  master  micro-  - , . 

miniature  station  . . -.  . 

♦ Eatabllshing  EV  antenna  repair  area  # 

\ ^ .V 

♦ Small  machine  shop  available  with  area  for  expansion 

* Packing  capability  with  expansion  area 


'-  m-* 


i } 


1 

* Parts  store  room 

i'  • 

* Technical  library  - now  being  established 

* Parking  available  for  approximately  90  cars  ^ 

* Shipping  and  receiving  area  •-? 

Present  physical  plant  can  accoiomodate  approximately  a lOO-man  work 
force  on  a one-shift  basis.  This  capability  could  easily  be  expanded 
to  a 300-iian  force  by  tri-shif ting. 

Limitations: 

(1)  Cannot  handle  large  antennas  or  equipments  due  to  limited  lift 
capability. 
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( Figure  ) 1 
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In  FY  72  Coi^croller  DARCOM  established  a productivity  measurement  system  by 
developing  efficiency  index  with  data  extracted  from  the  CAOMAF/Budget  (Conmand 
Analysis  of  OHA  Fundlna)  system  and  other  selected  functional  reports.  In  addition* 
Kethods  and  Standards  (MiS)  Program  data  (formerly  Defense  Integrated  Management 
Engineering  System*  DIICS)  were  also  included  to  obtain  a composite  score  reflecting 
how  available  resources  are  used  by  each  manager.  This  measurement  of  efficiency  was 
developed  into  a composite  efficiency  index  for  the  depots  and  coninodity  conmancb  within 
QARCON.  Five  basic  data  elements  (OKA  dollars*  workload*  manyears*  work  standards* 
actual  timed  work)  from  the  four  functional  areas  of  supply*  maintenance*  base  operations 
and  procurement  were  used  to  develop  productivity  indices  of  dollars,  manpower*  and 
KIS  performance  efficiency.  These  in.lices  were  then  combined  into  a conposite  efficiency 
Index  which  measured  each  activity  over  tine  against  itself  and  automatically  iwighted 
each  functional  area  to  the  total  resources  of  the  activity.  Effective  in  FY  76*  the 
K&S  data  has  been  eliminated  from  the  system. 

The  co^)osite  index  compares  like  activities  with  respect  to  a base  year  to  deter- 
mine which  activities  are  low  in  terms  of  efficiency  improvement.  Once  these  activities 
are  Identified*  an  attempt  is  made  to  determine  the  strengths/weaknesses  idiich  may  be 
reinforced  or  improved  to  increase  the  activity's  productivity.  This  concept  has  been 
working  well  at  the  depots  a.id  in  several  of  the  functional  areas  in  the  comnodity 
coanands.  However,  the  procurement  directorates  of  the  Major  Suoordinate  Coninands 
(MSC's)  have  not  felt  the  present  productivity  index  relating  their  output  to  input  is 
a true  measure  of  the  perfornance  of  the  central  procurement  offices. 

B.  Problem. 

The  output  measure  for  the  workload  of  the  procurement  and  production  directorate 
of  an  activity  is  given  in  terms  of  procurement  actions  and  procurement  line  items 
processed.  The  input  measure  is  given  in  terns  of  manyears  and  cost  expended  identified 
In  the  Arny  Management  Structure  Code  by  PE  721113  to  include  .1*  .2*  and  .3  categories; 
that  is*  manyears  and  cost  expended  in  procurement  operations*  contract  aitoini  strati  on* 
and  quality  assurance.  Herein  lies  the  problem  as  judged  by  the  procurement  activities. 

The  output  of  a central  procurement  office  is  being  measured  in  gross  ntinbers  so 
that  the  complexity  of  the  output  (a  procurement  action)  is  not  taken  into  consideration. 
If  a M&S  Performance  Efficiency  Index  had  been  or  could  be  developed  for  the  procurement 
activities*  then  perhaps  some  work  standards  for  different  methods  of  procurement  might 
have  been  introduced.  However,  as  it  is  yet  not  developed  and  there  is  some  question 
as  to  the  feasibility  of  its  ever  being  dieveloped*  the  present  procurement  productivity 
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Index  does  not  take  Into  consideration  the  aaount  of  inan*effort  required  for  different 
■ethods  of  procureoent.  dollar  values,  and  types  of  contracts. 


Secondly,  It  is  felt  that  neither  procurenent  actions  nor  procurement  line  Items 
processed  are  the  proper  output  measures  to  be  compared  with  the  Input  Incurred  inder 
contract  atWnIstratlon  and  quality  assurance.  There  Is  a good  deal  of  acMnlstratlon 
after  contract  award  idilch  cannot  be  charged  against  the  procurement  action.  Addition- 
ally, since  quality  assurance  Is  a product  assurance  function.  It  should  not  be  Included 
with  the  procurement  workload. 


C.  Objecti ve. 


r • ^ > 


The  objective  of  this  report  Is  then  to  develop  a productivity  Index  taking  Into 
®®*s1derat1on  the  amount  of  mao'^ffort  expended  for  different  methods  of  procurement 
and  types  of  contracts;  and  to  examlns  ways  of  measuring  the  output  associated  with  the 
amount  of  man-effort  Involved  In  the  administration  of  a contract. 

0.  Scope  and  Method.  _ . . ^ 

The  AHCRP-127  central  procurement  workloading  report  presently  records  procurement 
work  directives  (PUD's)  as  to  method  of  procurement  and  dollar  threshold  and  as  to 
idiether  thty  are  cancelled  or  transferred  prior  to  award.  Since  ANCIIP-1Z7  Is  already 
In  existence  and  provides  a source  for  reliable  procurement  productivity  data,  the 
productivity  model  has  been  developed  around  this  central  procurement  wortloading  report. 

Although  time  standards  have  been  considered  for  different  methods  of  procurement 
none  are  presently  being  used  as  established  work  standards.  Since  traditional  work 
Kasurement  methods  do  not  appear  to  apply,  a different,  wre  subjective  approach  has 
been  considered.  Assuming  It  Is  possible  to  subjectively  weigh  the  WD's  recorded  In 
the  AHCPP-127  report  as  to  anoint  of  man-effort  necessary  to  award,  then  a meaningful 
output  measure,  finctlonally  oriented  to  the  procurement  workload,  mqr  be  considered. 

Someone  with  years  of  varied  experience  In  procurement  may  have  some  feeling  as 
to  the  degree  of  man-effort  needed  to  accoapllsh  the  award  of  a PWO  and  assign  weights 
to  PWD's  commensurate  with  the  amouit  of  work  Involved.  The  weighted  PWD's  rather  than 
s1^>1y  total  number  awarded  will  then  account  for  the  output  of  a procurement  and  pro- 
duction directorate.  A survey  of  45  experenoed  contracting  officers  (those  In  grades 
GS-12  and  above  with  at  least  10  years  experience  In  procurement)  was  made  to  estimate 
the  respective  welghU  of  the  PWD's.  After  a statistical  analysis  was  performed  on  the 
wel^ts  assigned  by  the  contracting  officers  who  participated  In  the  survey,  the  delphi 
technique  was  further  applied  to  the  mean  weights  to  minimize  the  variability  Inherent 
In  the  survey.  Since  no  established  work  standards  are  available  for  different  methods 
of  procurement  and  typ*s  of  contracts,  actual  manhours  per  type  of  WD  are  not  directly 
measurable  outputs;  however,  this  does  not  mean  that  an  estimated  weir  t according  to 
amount  of  man-effort  required  cannot  be  assigned. 

In  the  case  of  Identifying  output  for  contract  administration,  the  ANCRP-127  central 
procurement  workloading  report  may  again  be  used.  The  127  report  has  a section  which 
tracks  the  nuober  of  contractual  type  documents  being  administered  for  production  and 
contract  administration  actions.  If  weights  are  assigned  to  each  type  of  document 
(contracts.  Board  of  Awards  and  Purchase  Orders)  then  the  total  nunber  of  weighted  doc- 
laents  may  be  considered  as  output  to  correspond  to  the  Input  provide  by  the  .2  category 
of  P£  721113,  contract  administration. 


This  concept  of  weighted  PJD's  and  types  of  contractual  documents  as  output  measures 
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and  PE  721113.1  and  .2  as  Input  Is  used  In  this  report  to  detemlne  the  procurenent 
productivity  Index. 

HEASURING  PaOOUCTIVI*^  » j-t  . i . -s  ^.vy-n 

- ' * " ^ *'  • ' i **  . • 

A.  Present  Itethod  - 

Tfh®  Index  In  the  ftnctlonal  area  of  procureeient  Is  the  avera<fe  of  the  nanpoifer 
and  dollar  Indices.  The  Input  factor  for  nanpoMer  Is  manyears  expended  under  the  Arm 
ManageKnt  Structure  Code  PE  721113  (AR  37-100-FY)  to  Include  the  .1,  .2,  and  .3  cate- 
gories of  procurement  operations^  contract  administration  and  quality  assurance,  respect- 
ively. The  constant  (Inflated)  dollars  expended  for  the  721113  code  by  an  activity  Is 
the  other  Input  measure. 

The  output  measures  are  procuroment  actions  and  procurement  line  Itew  processed. 
Mhat  constitutes  a procure«nt  action  is  defined  In  AR  37-100-FY  under  the  perfonmnce 
factor  for  procurement  operations  but  essentially  they  m^y  be  considered  as  procurwnt 
Mrk  directives  (PWD's)  tdiich  have  been  awarded.  Procurement  Line  Items  Processed 
(LIP'S)  are  the  summation  of  the  nunber  of  PWD's  awarded.  PWD's  cancelled,  mid  PWD's 
transferred. 

Essentially,  five  steps  are  Involved  in  the  computation  of  the  Index: 

a.  Identification  of  outputs  and  inputs. 

b.  A workload  relationship  of  line  items  processed  to  procurement  actions 
given  in  percentages  (such  as  40/60.  50/50,  30/70)  which  each  comnand  assigns  Itself. 

c.  CbnpuUtlon  of  the  weighing  factor  where  the  base  year  inputs  (mar\years 
expended  or  constant  dollars)  are  divided  by  the  base  year  outputs  (wori:1oad  in  terms 
of  LIP'S  and  PA's)  to  obtain  manyears  per  action  or  similarly  dollar  per  action. 

d.  The  outputs  of  each  year  are  then  multiplied  by  the  relationship  of  LIP's 
to  PA  s and  by  the  weighing  factor  In  manyears  which  were  both  computed  In  the  base  year. 

e.  The  weighted  outputs  for  each  year  are  then  divided  by  the  actual  Inputs 
(manyears  exp^.nded  and  constant  dollars).  The  mathematical  form  of  the  present  procure- 
ment perfonance  Indicator  is  11  iterated  In  Figure  1. 
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The  ex«^1e  in  Figure  2 uses  the  wthejoaticel  relationship  of  Figure  1 to  ej^lain  the 
coopi’^^ations  involved  in  a-riving  at  a prooireaent  productivity  index  for  a ■ajor 
subordinate  coflaand.  The  Dutputs  for  each  succeeding  year  after  the  base  year  are  nilti- 
plied  by  the  ueighting  factors  established  in  the  base  year  72.  The  weighted  outputs 
fo**  each  year  are  then  divided  by  the  actual  inputs  to  arrive  at  productivity  indices 
for  nanpower  and  dollars.  The  wanpower  and  dollar  indices  are  then  a«‘  '^aged  to  obtain 
'^he  procurenent  index.  The  procurenent  index  is  equal  to  ric  for  the  jase  year  and  then 
will  be  greater  than^  less  than,  or  equal  to  one  in  the  following  years,  indicating  an 
increase,  decrease,  or  no  change,  respectively,  in  procureaent  producti  ity.  The  sane 
procedure  is  used  to  calculate  indices  for  the  functional  areas  of  supply,  wi^ntenance. 
and  base  operations.  These  indices  are  then  ood)ined  into  the  cxMposite  efficiency 
index.  Hote  that  although  a weighted  output  is  coiqiuted.  it  is  not  weighted  to  the 
anount  of  nan-effort  involved  as  to  the  method  of  procurenent  or  type  contract  ei^loyed 
to  arrive  at  a procurenent  action  In  addition,  there  is  a double  counting  involved 
since  the  procurenent  line  itens  processed  includes  PUD's  awarded  which  are  equi  ilent 
to  procurenent  actions. 

Whereas  the  present  nethod  employed  by  Comptroller  to  arrive  at  a productivity  index 
nay  work  well  for  depot  maintenance  operations,  which  are  greatly  conducive  to  K&S 
usage  since  they  are  production  line  type  operations,  providing  nsaningful  data  down 
to  the  component/subasseshly  level.  major  subordinate  cowaand's  procurement  operations 
measure  only  simwary  level  data:  i.e.,  mmber  of  procureoent*  actions  and  ramber  of  line 
items  processed.  In  addition,  some  of  the  input  used  to  arrive  the  procurenent 
productivity  index  does  not  correspond  to  the  proper  performance  rector.  The  input 
used  under  the  .2  and  .3  codes  (contract  adeini  strati  on  operations  and  quality  assurance 
for  central  procurement  activities)  have  "contracts  requirement  production  action"  and 
"dollar  value  of  material  Inspected  and  released  for  shipment",  respectively,  is  their 
performance  factors,  not  procurement  actions  and  procurement  line  itens  processed. 

. Therefore,  in  order  to  be  fair  to  the  central  procurement  activities  a new  pro- 
curenent productivity  index  should  be  eoployed  which  relites  the  pi3per  output  to 
the  input  provided  by  the  .1,  .2  and  .3  codes  of  AR  37-100-FY  and  measures  the  nan-effort 
eaployed  by  each  different  nethod  of  procurenent  which  HAS  has  been  unable  to  do. 

B.  New  Method. 

1.  Introduction.  This  report  concentrates  on  the  reliability  and  quality  of  the 
output  measure,  ^tput  should  be  able  to  be  easily  counted  consistently  year  after  year. 
It  should  be  nitually  exclusive  of  any  other  output  so  that  double  counting  will  be 
avoided.  It  should  be  the  final  product  (or  an  intermediate  product  contributing  to  the 
final  product)  of  a significant  group  of  workers  whose  tine  and  costs  can  be  directly 
identified  with  the  output.  In  a service  organization  it  is  often  difficult  to  identify 
with  the  final  product.  For  exaiple.  in  a maintenance  hangar  it  is  easy  to  identify  the 
final  product  as  a repaired  aircraft;  however,  it  is  nore  meaningful  to  distinguish 
between  aircraft  such  as  ‘onbers,  filters,  trainers,  and  helicopters.  Fach  different 
type  aircraft  requires  more  or  less  tine  and  skill,  hence,  an  atteiw>t  should  be  nade  to 
differentiate  and  weight  the  output  according  to  the  effort  needed  to  produce  it.  This 
concept  holds  true  for  the  workload  in  a procurenent  directorate  as  well  as  in  a nin- 
tenance  hangar. 

Not  all  procurement  work  directives  (PWD's)  entail  the  sane  amount  of  manhours 
to  award.  The  amount  of  manhours  to  award  a contract  or  process  a PUD  may  be  explained 
by  the  distinction  of  such  cooplexity  factors  as  thedollar  amount,  the  method  of 
procurement,  and  the  type  of  contract  applied  for  the  proposed  awani.  Therefore,  the 
use  of  total  nusber  of  PWD's  processed  as  an  output  measure  fails  to  differ  ntiate  as 
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• EXAMPLE  COMPUTATIONS  OF  PROCUREMEMT  IMDEX 

Figure  2 


to  rJ>«  quontity/quallty  of  ■on-effort  required  for  each  ^ype  of  PWD. 

2.  ^currnent  Operations.  The  perforaance  factor  which  Is  felt  to  be  aore 
representative  of  the  workload  output  to  a central  procurement,  office  Is  procurevnt 
line  Items  processed.  Procurement  line  Items  processed  are  made  up  of  procurennt 
work  directives  processed  to  award»  cancelled  and  transferred.  These  PMD's  cover  the 
entire  workload  area  for  procurement  operations. 

••  Procuremwt  MOrk  Directives  Awarded.  Nhlle  a PWD  which  has  been  awarded 
Is  equivalent  to  a proemment  action,  gross  nuohers  of  procurement  wort  directives  by 
themselves  do  not  ejqilafn  the  amount  of  work  Involved  in  each  PWD.  Currently.  ANCRP-127 
Central  Procurement  Workloading  Report  lists  the  nuWber  of  PWD*s  awarded  by  dollar 
^reshol^  and  vthods  of  procurement.  A program  change  by  the  Amy  Logistics  Managemt 
^tMs  Agency  (ALM5A)  Is  planned  to  Include  type  of  contract  In  the  current  report  of 
the  Procurement  and  Production  Subsystem  of  the  CoModIty  Coonand  Standard  System  (CCSS). 

There  are  no  generally  established  standards  for  the  nunber  of  nanhours  required 
to  award  a PWD;  therefore,  manhours  per  type  of  PWD  do  not  exist.  However,  this  does 
not  preclude  the  possibility  of  subjective  weigiting  of  PWD's  according  to  the  avunt 
■an-€ffbrt  felt  to  be  Involved  In  their  aMard.  A survey  was  Bade  up  geared  to  the 
yiW-127  report.  Eight  aethods  of  prooirenent  were  listed  in  survey  categorized  by 

value  (under  $10,000.  $10,000  and  over)  and  type  of  contract  (fixed  price  or  cost). 
Experienced  personnel  throughout  DARCON  were  asked  to  weight  the  different  methods  of 
procurement  according  to  dollar  size  and  type  of  contract.  Ten  percent  of  the  weights 
were  ewiuded  - five  percent  at  each  end  of  the  distribution.  After  a sutlstical 
mal^ls  of  the  data,  the  delphi  technique  was  used  to  refine  the  mean  weights  so  as  to 
minimize  the  variability  Inherent  In  any  survqy  of  this  type. 


. , , ^curement  Work  Directive  Cancelled.  Oftentimes  a PWD  requires  a great 

«a1  0^  mmi-effort  but  does  not  result  in  an  award.  That  Is  to  say.  a requirement  Is 
given  to  procurement  personnel  for  a particular  Item,  and  the  personnel  proceed  to 
procure  the  Item.  After  all  the  necessary  work  has  been  done  and  all  that  Is  left  Is 
to  sign  a contract,  the  rtvpilrement  Is  cancelled  for  reasons  which  are  no  fault  of  the 
procurement  personnel  (lack  of  funds  or  the  requirement  Itself  Is  cancelled).  The  Mount 

Is  still  part  of  the  procurement 

workload.  Naturally  all  PWD's  do  not  proceed  In  time  up  to  the  point  of  award  before 
they  are  cancelled,  but  may  Instead  be  cancelled  after  5i.  25X,  50t,  or  75X  of  the 
total  nan-effort  to  award  has  been  expended. 

Cancelled  PWD's  may  represent  a significant  amoimt  of  a procurenent  directorate's 
wrkload.  An  obvious  answer  to  the  problem  of  cancelled  PWD's  Is  to  better  manage  the 
i^uirements  and  budgeting  of  funds.  However,  the  control  of  these  functions  Is  external 
to  the  procurenent  directorate  and  it  is  not  appropriate  to  measure  a procurement 
directorate  for  something  over  which  in  nost  cases  they  have  no  control.  By  the  saoe 
token  it  would  be  Inappropriate  to  allow  the  total  nuBt)er  of  cancelled  PHD's  to  be 
converted  as  output  If  only  5X,  or  25X  of  the  amount  of  man-effort  needed  to  award  Is 
expended.  Hence,  based  on  a review  of  cancelled  PWD  data  and  perceptions  of  convnand 
personnel,  a figure  of  SOX  Is  used  to  balance  the  expenditure  of  the  ti*  between 
^se  cancelled  PWD's  requiring  more  man-effort.  If  to  that  point  In  time  until  a 
PWD  Is  cancelled,  the  amount  of  work  Involved  In  each  Is  still  determined  by  the  cooplex- 
Ity  factors  of  tee  method  of  procurement,  dollar  value,  and  type  of  contract,  then  each 
should  be  weigited  equivalently  to  the  one  which  Is  awarded.  However, 
only  50X  Or  those  PWD*s  reported  as  cancelled  with  a designated  method  of  procurenent 
and  type  of  contract  will  be  counted  as  a weighted  output. 


The  current  AMCRP-127  Central  Procureaent  Workloading  Report  lists  the  nunt)er  of 
cancelled  PWO's  with  a designated  dollar  threshold  and  aethod  of  procureaent.  A prograa 
change  is  planned  to  Include  type  of  contract* 

* c.  Procurement  Work  Directives  Trwfcrred.  The  third  categoirv  of  procureaent 
line  iteas  processed  Ire  procureaent  work  directives  transferred  either  as  Military 
Interdepartaental  Purchase  Requests  (NIPRS)  to  other  sendees  or  other  similar  doctaants 
utilized  to  transfer  the  procurement  requirements  to  other  cooaands.  Installations,  or 
activities.  In  some  cooiaands  the  number  of  PWD*s  transferred  aake  up  a significant 
portion  of  their  workload.  The  greater  the  dollar  value  the  more  aonitoring  is  required. 
Although  the  aaount  of  aan-effort  required  Is  not  as  great  as  that  necessary  for  Mrd 
of  a PUD,  tiae  and  effort  are  expended  and  hence  should  be  weighted  accordingly  in  the 
output  portion  of  a procurement  productivity  nodel.  Those  conaands  whose  workload 
contain  a slgilflcant  amount  of  transferred  PWO's  were  consulted  for  comparison  between 
PWO's  awarded  and  transferred.  Their  co— ints  were  evaluated  and  weights  were  assigned 
eigiloylng  the  delphi  aethod  by  personnel  external  to  the  oonnands  consul te<L  The 
current  AMCRP-127  Central  Procurement  Wortloading  Report  lists  the  niaber  of  PWO's  trans- 
ferred by  dollar  threshold. 

2.  Contract  Administration.  A great  deal  of  aan-effort  Is  expended  try  a procure- 
ment directorate  in  the  area  of  contract  administration.  In  the  present  productivity 
model,  this  man-effort  has  credited  to  the  procurement  directorate  In  the  form 

of  Input;  I.e.,  It  Is  recot  ..d  In  the  .2  category  of  the  Arwy  Management  Structure 
Code  PE  721113,  and  used  along  with  the  man-effort  ejq)ended  In  the  .1  and  .3  categories. 
However,  the  output  performance  measure  used  to  explain  the  .2  Input  Is  the  same  as 
that  to  explain  the  .1  Input  for  procurement  operations;  I.e.,  procurement  line 
Items  processed  and  procurement  actions.  A great  deal  of  work  Is  expended  after  the 
award  of  a contract  which  Is  not  explained  by  present  performance  measures.  If  any 
procurement  productivity  model  Is  to  be  meanln^ul,  an  atteipt  should  be  made  to 
incorporate  a reliable  performance  factor  which  credits  that  percentage  of  the  procure- 
ment directorate's  workload  dealing  with  contract  administration  to  the  output  portion 
of  the  iBodel.  If  this  cannot  be  done,  then  the  12  category  of  PE  721113,  Contract 
Administration,  should  not  be  Included  as  Input  as  Is  presently  done. 

A performance  factor  for  the  .2  category  Is  defined  In  AR  37-lCX)-FT  as  "contracts 
requiring  production  action,"  btJt  It  Is  not  used  In  the  present  model.  However,  If  It 
Is  used,  the  same  argument  exists  here  as  did  for  procurement  actions,  and  that  Is  that 
all  contracts  do  not  require  the  same  amount  of  work  and  hence  should  not  be  treated 
siwply  in  tenns  of  gross  numbers.  There  are  some  simple  documents  to  administer  such 
as  purchase  orders,  delivery  orders,  basic  ordering  agreements  (BOA's),  and  then  there 
are  contracts  such  as  production  versus  R&O.  The  amounts  of  man-effort  Involved  In 
administering  these  docunents  will  differ.  Therefore,  It  Is  logical  to  weight  each  type 
of  document  depending  on  the  expenditure  of  man-effort  required  to  administer.  The 
AMCRP-127  Central  Procurement  Workloading  Report,  lists  the  nunter  of  purchase  orders, 
delivery  orders,  BOA's,  and  contracts  both  In  productions  actions  and  contract  manage- 
ment actions  taken.  Weights  for  each  type  dociment  were  solicited  from  contracting 
officers  throughout  OARCOM.  Their  ronnents  were  recorded,  evaluated  and  then  subjected 
to  the  delphi  technique. 

3.  Quality  Assur^ance.  The  roan-effort  expended  under  the  .3  category  of  PE  721113 
should  not  be  evaluated  by  the  performance  factor  for  procurement  operations  or  contract 
administration.  The  opinion  of  this  report  Is  that  any  Input  expended  In  the  .3 
category  of  quality  assurance  should  be  accountable  by  the  Directorate  for  Quality 
Assurance  and  not  Procurement  and  Production,  since  quality  assurance  Is  a product 
assurance  function.  In  addition,  the  PU’  Directorate  does  not  have  functional  control 

over  quality  assurance  personnel,  and  no  output  measures  to  account  for  their  Input, 
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hence,  quail ^ assurance  should  not  be  included  with  the  procurement  workload. 
C.  Differences  Between  Present  and  New  itettod.  : . . 


The  primary  difference  between  the  new  method  for  measuring  productivity  and  the 
present  Is  the  use  of  weighted  PWD's  awarded,  cancelled,  and  transferred  in  place  of  - 
total  line  items  processed,  and  the  use  of  weighted  contractual  typc^docuKnts  Instead 
of  toUl  procureient  actions.  In  addition,  the  only  input  which  will  be  used  in  the 
pi^uctivity  model,  both  in  mar^ears  and  dollars,  will  be  that  providedMn  the  .1  and 
.2  categories  of  PE  721113.  The  quality  assurance  input,  .3  category,  will  be  dropped, 
and  there  will  be  no  further  need  for  any  relationship  factor  of  LIF's,  to  PA's  as 
thero  is  in  the  present  indicator  (recall  figure  1).  The  proposed  performance  indicator 
^11  take  on  the  form  in  figure  3,  where  PWO'sa,  W0*sc,  PWD'st#  «re  procurement  work 
directives  awarded,  procurement  work  directives  cancelled  and  procurement  work  directives 
transferred,  respectively;  and  KO  is  the  nunber  of  contractual  docimmnts  having  produc- 
tion and  contract  management  actions  applied  thereto.  < 


PUO'Sj^  PWD'sc  PWO's^  ^eighting  Factor  In  Itayear^ 


• f t 


Kanyears 

t [yei0t1m9  Fsetor 


Mmiyears 


In  Manyear^  j 


i.I  v: 


' »c.r'v»,h 


Ir«  PWO'sa  ♦£i<t  PWD'sc  ♦25*^  PWD's^  jwelghtlng  Factor  In  Oollar^ 


Dollars 


i- 


CSijsiL  fwelghting  Factor  In  DplTarQ 


OBllars  • 

PROPOSED  PHOCUREfCIT  INDEX  - Figure  3 

P**  Figure  4 Illustrates  the  new  methodology  of  the  proposed  procurement 

Index.  Note  that  It  Is  now  possible  to  separate  and  analyze  output  and  input  with 
respect  to  procuremt  operations  and  contract  ackalnl  strati  on;  In  addition,  the  dollar 
breakout  is  much  more  visible.  Consequently.  If  desired,  separate  productivity  Indices 
may  be  kept  on  procurement  operations  and  contract  administration  and  be  used  by 
management  In  the  P4P  Directorates. 

Apart  from  serving  as  Input  to  the  Comptroller  composite  efficiency  index,  this 
m^l  my  be  used  by  the  P&P  Directorate  action  officers  as  a tool  In  the  management 
objectives.  The  AMCRP-127  report  maintained  by  personnel  in  the 
W Directorate  serves  as  the  primary  data  base  for  the  model.  Only  Input  data  for  the 
*11ar  productivity  Index  requires  data  provided  by  the  Comptroller  Approved  Operating 
Bud^^  CSCFA-218  Report.  All  other  data  for  both  manpower  and  dollar  Indices  Is 
provided  by  AMCRP-127.  The  following  Table  I explains  where  the  data  Is  found  In  the 
report  and  idiat  portions  are  presently  In  the  CCSS  system.  In  addition,  the  model 
mil  make  readily  available  the  distribution  of  wrkload  as  to  procurement  operations 
(broken  out  as  to  PWO's  awarded,  cancelled,  and  transferred)  and  contract  administration 
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EX/<HPLE  ILLUSTRATION  OF  NEW  PROCUREMCNT  PROOOCTIVITY  INDEX 

Figure  4 
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Procuramant  Operations  • 721113.1 
(In  ALPHA  System) 
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Awarded 
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(Initially,  the  dollar  breakout  was  to  be  under  $10,00  and  over  $10,000*,  however, 
ivon  critique  of  the  study  several  reviewers  thought  a better  weight  distribution  could 
be  developed  If  the  dollar  breakout  consider  $10,000  to  $100,000,  and  over  $100,000. 

The  reason  this  breakout  was  not  ei^loyed  originally  was  that  the  /INCRP-127  report  only 
considered  Method  of  procureoents  under  $10,000  and  over  $100,000,  and  It  was  already 
autoaated  In  CCSS.  However,  since  the  addition  of  the  third  dollar  category  Is  oonsid* 
ered  to  be  nost  beneficial  to  the  productivity  nodel.  It  will  be  Included  In  the  ANCPf- 
^7  report  and  hence  In  CCSS.  When  this  report  Is  published  by  APRD  the  weight 
distribution  Is  to  Method  of  pAtcureMent,  dollar  threshold,  and  type  of  contract  will 
be  Included.) 
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p'*""*"9  represents  an  on-going  and  extremely  Inooi-tant 
amicable  to  all  eleaents  of  the  material  acquisition  cycle 

position  of^S^^tS'd^  SfKn7®"i  ®^5^*"**  procurement  managers  In  the  challenging 
-2nt ‘ “eal  eff«t1vely  with  a more  sophisticated  and  demanding  procure^ 
ment  workload  while  generally  faced  with  declining  manpower  resources.  ^ ^ 

Aaeiicr7D!awiL^‘'‘*K‘  Senrlces  (DCAS)  of  the  Defense  Logistics 

«"Phas1s  on  manpower  planning  In  order  to  provide  the  most 

7k.  **"’1*  ***  effective  manpower  management  certainly  encovasses 

wni*^Itt«ot^7nustrSS‘th!Vh7ii’''^"®K"7^  agwies  and  activities,  this  prSntatioo 

a large‘SS  ?h1s  «se!“Iiri^s2n'SSlS 

I wll^Stel  application. 

SLIDE  #1  OH 

(1)  The  organizational  relationship  of  DLA  and  DCASR  Boston, 
then  I^iiirbriefly^^^*^*”^’  Boston's  organization  and  Its  production  element. 

(3)  Give  an  encapsulated  review  of  the  major  production  functions  performed. 


...... mpwci^ui  cn^iii.  u^in^  can  1C 

methodology  and  Its  application. 
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production  directorate  In 

volume. 
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of  workload  and  related 

development  of  our  current 
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SLIDE  n ON 

The  Director  of  the  Defense  Logistics  Agency.  Lieutenant  General  M-  M.  Vaughan.  USA. 
has  tuenty-four  najor  field  activities  reporting  to  h1«:  6 Supply  Centers,  4 Depots. 

5 Service  Centers,  and  9 Defense  Contract  Atelnistratlon  Services  Regions.  The  De^ty 
Director  for  Contract  Adelnistratlon  Services  Regions  Is  Major  General  Vincent  H.  Ellis. 
USA.  The  Boston  Region's  Coanander  Is  Captain  T.  J.  Mulligan.  USN,  the  Director  of 
Production  Is  Mr.  Ralph  Kabureck. 
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SLIDE  n OH 

OCAS  Regions  are  structured  In  a straight  fonaard  tMO  level  organi zational  yncept 
consisting  of  the  region  headquarters  and  Its  subordinate  field  operating  eienents.  The 
field  elements  consist  of  Defense  Contract  Aihrinlstratlan  Services  Managenent  Areas  and 
Plant  Representative  Offices  referred  to  as  -OCASMs-  and  -DCASPROs'.  Both  typn.  of 
operational  field  activities  are  assigned  sinilar  nlsslons:  have  equal  oroanlzatlnnal 
status  with  their  respective  coamander  reporting  directly  to  the  region  cnmnander:  and 
differ  nalnly  In  regard  to  geographic  territory  assigned  and  the  maber  of  contractors 
with  which  they  Interface. 
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SLIDE  #4  ON 

OCASR  Boston  has  twelve  field  activities  - 7 DCASWS  and  5 OCASPROS-  Each  of  these 
offices  has  an  Integral  OCAS  production  aanagement  operational  eleaent  1«  the  fora  of  a 
production  division. 


I ' 


SLIDE  IS  ON  , 

DCASR  Boston,  as  a prlnary  field  level  activity  of  the  Defense  Logistics  Agency,  Is 
responsible  for  all  contracts  delegated  by  procurement  agencies  for  contract  adalnlstra- 
tion  services  within  upstate  New  York  and  the  six  New  England  sUtes.  It  serves  procuring 
activities  principally  through  the  field  management  of  contracts  including  the  performance 
of  those  administrative  functions  defined  in  ASPR  1-406. 


SLIDE  #6  ON  • . 

Briefly,  the  primary  functions  supporting  the  DCAS  production  mission  are  carried 
out  by  industrial  specialists,  engineers  and  other  professionals.  These  people 
participate  In  pre-award  surveys  of  potential  contractors  which  determine  whether  the 
offeror  Is  qualified  to  produce  In  accord  with  the  terns  and  conditions  of  the  solicita- 
tion before  a contract  Is  awarded.  Once  the  award  Is  made,  surveillance  is  maintained 
over  the  contractor's  progress  toward  meeting  delivery  dates;  delays  are  reported  and 
any  corrective  action  that  Is  needed  is  taken.  When  necessary  contractors  are  assisted 

in  obtaining  priorities  for  critical  materials.  In  addition  a digest  Is  published 
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annually  that  facilitates  the  exchange  of  wnufacturing  cost  control  Ideas  among 
defense  contractors. 


Surveys  are  conducted  tc  assure  that  government  owned  plant  equipment  In  use  by 
defense  contractors  is  efflcl^tly  utilized  and  that  adequate  maintenance  Is  provided  V ‘ 
for  government  owned  plants. 


, 1 
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SLIDE  #7  OH 


The  production  organization  also  conducts  surveillance  to  assure  co^>11ance  with  « \ 

Environmental  Protection  Agency  regulations  In  regard  to  air,  noise,  and  water  pollution  ^ 7 
for  govemgmnt  owned  facilities  located  within  the  region. 

Surveys  are  performed  for  Industrial  preparedness  planning  requirements  from  ( ) 

claimant  activities  to  assure  an  adequate  Industrial  production  capability  In  the  event  ^ 
of  national  mobilization  or  a limited  war. 


A voluntary  program  with  Industry  to  maintain  energy  conservation  Is  accomplished  ( ) 

In  cooperation  with  the  Federal  Energy  Adalnlstratlon.  Department  of  Commerce,  Defense 
Contract  Audit  Agency,  state  and  local  goverments.  Included  In  OLA's  energy  program 
are  fuel  shortage  assistance  procedures  to  assist  contractors  In  obtaining  fuel  alloca-  , . 

tlons  adequate  to  meet  essential  defense  production  requirements.  Defense  Logistics  ■ / 
Agency  production  related  activities  Include  engineering  surveillance  and  assistance. 


SLIDE  *8  OH 


Evaluation  of  proposed  engineering  changes  and  maintenance  of  a value  engineering  , ) 

program.  Technical  support  to  program  managers  Is  provided  through  OCAS  surveillance  — 
of  cost/schedule  control  systems  program  contracts  and  special  emphasis  programs  and  . 
technical  analysis  of  cost  proposals.  . ~ . 


SLIDE  19  ON 

Other  services  Include  monitoring  by  Industrial  labor  relations  specialists  of 
actual  and  potential  strikes  at  contractor  plants:  coordinating  removal  of  critical 
defense  material  from  strike  bound  plants:  and  services  related  to  the  enforcement  of 
labor  standards  and  Industrial  manpower  requirements. 


SLIDE  110  OW 

Transportation  and  packaging  specialists  assigned  to  the  production  organization 
assure  on-time  shipments  of  damage  free  material  via  the  most  economical  means  of 
transportation.  Their  work  Includes  technical  assistance  to  contractors:  Issue  and 
control  of  Government  bills  of  lading,  and  other  transportation  doctaentatlon. 


( ) 
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The  Boston  DCASR  has  sow  2,900  contractors  of  which  1,700  are  active  In  any  given 
period,  with  approxlwtely  30,000  contracts.  The  face  value  of  these  procurewnts  Is 
12  billion  dollars  with  an  unliquidated  value  of  S billion  dollars  and  a dally  disburse* 
wnt  of  12.5  Billion.  The  DCASR  wnnlng  level  Is  currently  2,355. 
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You  will  notice  that  the  production  statistics  are  In  all  cases  less  than  the 
region's  statistics.  This  Is  due  to  the  fact  that  not  all  contracts  ackilnlstered  by 
the  region  have  production  a<ta1n1strat1on  services  delegated. 

The  production  directorate  Is  presently  authorized  362  civilian  positions.  The 
contract  population  under  production  surveillance  Is  22,000  with  1,500  active  contractors 
with  an  obligated  value  of  9.7  Mlllon  and  an  unliquidated  value  of  3.5  billion. 
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Of  the  362  authorized  positions*  there  are  soae  37  distinct  occupational  disciplines 
and/or  separate  grade  levels  irlthin  our  production  organization*  which  are  distributed 
aaong  12  operational  organizations  and  the  region  headquarters  staff*  which  Illustrates* 
to  sone  degree*  the  cooplexlty  and  challenge  of  effective  aanpower  aanageaent. 

personal  experience  Includes  service  In  both  a large  area  office  and  two  plant 
offices.  During  ny  tenure  at  these  offices*  It  becaae  evident  to  me  that  certain 
distinct  differences  In  the  nature  of  the  work*  due  to  the  typos  of  contracts  being 
adilnistered*  was  apoarent.  Our  product*  production  services  to  our  custoaers*  and 
volune  of  work  Is  no  doubt  driven  by  the  nunber  of  contracts  we  receive.  This  priaary 
workload  Indicator  did  not  In  my  opinion  reflect  the  degree  of  cowplexity  associated  with 
the  different  offices. 

Work  patterns  are  fonwulated  by  the  nature  of  the  contract  alx  and  associated 
contract  characteristics  of  each  of  the  two  operating  environaents*  area  and  plant  offices* 
idilch  appears  to  be  somewhat  of  a dlchotonor.  Area  offices  stressed  the  wide  variety  of 
coanodltles  and  large  nuid>er  of  contracts  which  they  had  to  a(ki1n1ster*  while  plant 
offices  eaphaslzed  the  higher  dollar  value  of  their  contracts  and  the  generally  more 
sophisticated  systems  or  sub-systems  Involved.  The  primary  workload  Indicator  was  the 
contract  population  for  each  office*  stratified  by  a categorization  system*  and  an 
associated  average:  A lower  one  for  an  area  office  and  a higher  one  for  a plant  office. 

Essentially*  the  DCASR  authorized  strength  Is  determined  by  the  DLA  Comptroller 
and  allocated  by  function  cost  account  to  the  Region  Commander  who  has  the  authority 
to  make  necessary  adjustments  or  resource  trade  offs  among  the  various  functional 
specialties.  The  charter  for  manpower  planners  then  Is  to  determine  workload  require- 
ments In  distinct  quantifiable  terms*  recognizing  the  different  operating  envirorvaents* 
yet  somehow  define  them  in  a way  which  would  be  conaon  to  both  areas.  The  available 


' “".***?  **  <11stribgt«l  to  the  twelve  operating  activities  and 

the  regional  staff  organization,  adjusting  theai  as  worhload  fluctuates. 

r- 

SlliSS  “sirJS.S.”"*''"*''  " ““  •"  . 

Because  of  the  nature  of  OCAS  business,  essentially  perfomlng  a variety  of 

2 servicing  the  contracts  we  receive  and  assume 

2r2'15“t2t'5tar’2  wSliiJ!  '•  “ “ 

ti-.rhif«r  dissecting  Its  eloenU  like 

?*  “?  •Mlyze  that,  rebuild  “typical"  contracts. 
■ents™nd^dlstr1butlwr**  locations  and  assess  their  Inpact  on  manning  require- 

Reco^lzlng  that  wch  of  our  record  maintenance  Is  contained  without  computer 

2[^ahTi?t^  \l  our  Information  storage  and  extrapolation 

upablllty  Is  greatly  expanded  vis  a vis  prior  manual  systems.  We  are  only  United  then 
by  our  data  systems  and  realistically  with  any  new  programwing  costs. 

planner,  I have  had  an  opportunity  to  reflect  upon  these  conditions  and 
^l^nges  and  propose  what  I call  the  “pro^tlon  contract  attribute  system"  which 

• powea.  I believe  It  Is  a more  realistic  approach  with  greater 
a^  averaging  method  used  previously.  Additionally 
DCASMA/DCASPRO  commanders  and  division  chiefs  have  been  receptive  to  this  approach. 

OlA  has  a work  measurement  systaa  known  as  the  Defense  Integrated  Management 

5ill"^!l2  ?**^ri*4  r*4*r*2f  workload  and  resources  management,  which  I 

!!*  ir^^***:  ‘*®«s  not  conflict  with  the  DIMES  program;  conversely,  it  Id^lfles  work 
attributes  by  contract  distribution  idilch  should  enhance  the  prograa. 

The  six  characteristics  I will  address  focus  on  contract  aspects  that  will  enable 
us  to  analyze  and  recognize  office  location  workload  characterized  by  relatively  few 
contracts  but  requiring  significantly  aore  manpower  than  would  be  Indicated  by  the 
previous  method.  Analyzing  our  contracts  from  a production  standpoint  leads  to  the 
conclusion  t^t  t^  number  of  line  Items  Involved  and  frequency  of  delivery  schedules 
candidates  for  further  Investigation.  These  factors  are  what  we  "track 
actuar^rkload*^*^*^^^^”  surveillance  effort  and  therefore  are  Instrumental  in  defining 

DIMENSION  1 AND  2 

P*^^*cts  which  are  sorted  by  line  Item  and  this  data  Is 

dimension  of  our  analysis.  Let  us  abstract  for  our 
l^atlons  both  the  empiric  nuniber  of  line  Items  and  the  percent  dlstrl- 
oution  to  total  using  a homogeneous  or  family  grouping  for  the  DCASMAS  and  DCASPROS. 
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Frequency  of  schedule  fs  (a)  Perhaps  not  as  significant  an  elenent  If  we  assuoe 
that  once  the  line  Iten  Is  Identified  and  established  the  degree  of  difficulty  associated 
with  Its  surveillance  Is  of  considerably  less  value  than  the  line  Iten  Itself,  (b)  Our 
current  ADP  systen  does  not  Identify  Individual  schedules  which  are  past  due.  Future 
schedules  can  be  Identified  and  factored  with  future  lines  to  detenilne  schedule 
frequencies. 


DIMENSION  3 14 


ir’ 


Contract  population  Is  still  a valid  and  luportant  dimension  of  Individual  office 
workload.  Analyzing  It  closer,  the  dwell  time  or  contract  turnover  rate  Is  an  aspect 
which  also  requires  Investigation  for  a meaningful  and  comprehensive  workload  analysis. 

An  appropriate  analogy  from  a business  or  profit  oriented  perspective  is  If  we  owned 
our  Mn  retail  establishment  we  would  certainly  be  Interested  In  first  the  cost  price 
mark  up  by  product  and  second  the  shelf  turnover  rate.. .for  Its  effect  on  profits  and 
cash  flow,  not  unlike  the  shelf  turnover  rate  Is  the 'contract  turnover  rate.  Numerous 
coMon  fixed  work  efforts/tasks,  like  fixed  costs,  are  associated  with  every  contract. 

If,  In  fact,  there  are  significant  variations  from  location  to  location,  then  this  dimen- 
sion bears  Investigation. 

Contract  turnover  rate  can  be  measured  In  two  ways:  First  by  taking  all  contracts 
completed  Airing  a specified  time  period  and  dividing  by  the  average  number  of  contracts 
on  hand  for  that  same  period... preferably  « year  to  avoid  cyclical  peaks  and  valleys  and 
also  for  yearly  correlation  to  the  manyear  concept.  The  second  method  of  contract  turn- 
over rate  can  be  accomplished  by  a co^iuter  scan  of  aggregate  active  contracts  life  cycle 
In  calendar  days,  computed  from  the  effective  date  of  the  contract  measured  to  the  final 
delivery  date  or  forecasted  final  delivery  date,  whichever  1$  later,  divided  by  ihe  mmAer 
of  contracts  by  location  for  the  average  contract  life  time. 

Using  method  nuaber  one,  an  advantage  of  actual  history  Is  realized.  Utilizing 
method  nuaber  two.  an  Instataneous  cameras  view  Is  obtained;  however,  contract  extensions 
for  a variety  of  reasons,  could  possible  distort  actual  historic  patterns.  An  Initial 
correlation  study  of  these  two  methods  did  show  a deviation. 

I have  elected  to  use  method  number  one  because  of  t e higher  degree  of  confidence 
realized  with  the  historic  data. 

For  method  nimiber  one,  using  a one-ye  r period,  an  i».dex  can  be  developed  which  will 
allow  us  to  convert  the  actual  contracts  on  hand  to  the  equivalent  contracts  on  hand  due 
to  turnover.  For  example: 
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COffTRA-'TS  COMPLETED 

AVEftA&E  contracts  on  hand 


INDEX 


INDEX  TIMES  ACTUAL  CONTRACTS  ON  HAND  EQUALS  CONTRACT  EQUIVALENCY 
25,000  - 1.14 


1.14  X 24.000 


- 27,360 
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LET  US  TAKE  M OFFICE  WITH  THE  REVERSE  NUWER  FOR  A COMPARISON  ON  ICASUREO  EFFORT. 
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22.000  - .88 

.88  X 24.000  - 21.120 
27.360  • 21.120  - 6.24C 
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6,240 
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• 29.54X 
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IF  ALL  OTHER  WORK  OINENSIONS  HERE  THE  SANE  IN  OUR  HTPOTHETICAL  SITUAHON.  THIS  WOULD 
BE  A SIGNIFICANT  WORKLOAD  CHMMCTERISTIC. 

* ' 1 ‘ 
Reference  back  to  aetlnd  nuri>er  two.  the  contract  turnover  rate  Index  can  be 
detemined  by  dividing  the  average  contract  life  In  days  6y  365.  The  contract  equivalency 
1$  detemlned  by  Multiplying  the  Index  by  the  actual  contracts  on  hand,  -'or  ex«q>1e: 

SLIDE  #15  ON 

365  • 1.06  . ' ; . • ..  ; . 

m 


• • » 


1.06  X 24.000  - 25.440  AGAIN  TO  COMPARE  A REVERSED  STAHSTICAL  SITUATION 


365 

m.9 


.94 


.94  X 24,000  - 22.560 
25.440  • 22.560  - 2.880 


• 2,880 
22*560 


12.77X  AGAIN  ALL  OTHER  DATA  REMAINING  THE  SAfC  A SIGNIFICANT 
DEVIATION. 


Dinenslon  raaber  five,  the  dollar  value  of  the  contracts  Managed  Is  another 
iMportant  factor  to  be  considered  In  our  total  business  assessnent.  When  contract  value 
1$  assMiated  with  the  quantity  and  mix  of  line  Itens,  It  can  provide  an  Indication  of 
w ralative  coMplexIty.  particularly  when  coaparinq  our  operating  offices  location  to 
lo^tlon.  This  workload  characteristic  can  be  n*?astired  In  two  ways*  using  either 
Mllgated  or  unliquidated  contract  dollar  value.  I prefer  the  latter,  since  It  portr^vs 
the  balance  to  be  delivered  and  hence  relates  to  both  current  and  future  workload. 
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Th®  sixth  and  final  factor  to  be  considered  Is  the  nunber  of  active  contractor 
production  facilities  under  the  cognizance  of  a particular  office  location.  This 
diaenslon  gives  us  an  Indication  of  diversity  or  breadth  of  Morkload  uhen  again 
coaparlng  office  to  office.  A further  reflneaient  In  prograa  calculations  could  show 
distribution  by  density  uhlch  would  .’e  helpful  In  our  assessaent  considering  wide 
geographic  dIsUnces  between  facilities  versus  another  location  which  had  relatively 
confined  or  close  geographic  distribution. 

The  six  characteristics  which  I have  briefly  reviewed  are  conUlned  within  current 
•ADP  program  and  are  therefore  retrievable  without  establishing  new  daU  progrMS. 

In  order  to  uore  clearly  understand  the  conparatlve  relationship  of  the  twelve 
locations,  using  the  above  characteristics,  a aatrlx  containing  the  relevant  data  Is 
used  for  a specific  tiue  period  as  m would  In  accour/  mg  tenas  with  a balance  sheet 
or  cost  of  Manufacturing,  to  use  as  an  lllostratlon  of  the  analysis.  p 

4 ...  ^'^  **'^*  J'**^'*e,  we  win  use  30  April  77  statistics.  The  aalntenance  of  charactcr- 
Is^s  Is  rather  "busy".  As  you  can  see.  five  of  the  dimensions  have  both  the  eq>1r1c 
ntaber  and  tte  percent  to  total  colmns.  Is  order  to  more  clearly  show  the  effect  of  the 
characteristics  on  one  slide,  the  next  slide  will  drop  the  empiric  ni^iers. 


SLIDE  >17  Oh 


For  comparison  purposes,  family  grouping  of  area  offices  and  plant  offices  were 
structured  accordingly.  1.  e.  the  first  seven  offices  are  area  and  the  remaining  five 
are  plant  offices.  Now  our  collective  demographic  data  Is  ready  for  an  analysis  for 
workload  patterns. 


In  order  to  more  clearly  portray  some  specific  comparisons  the  next  slide  will 
explode  and  Isolate  four  dimensions  for  three  offices. 

SLIDE  118  ON 

Comparing  actual  contracts  with  the  contract  turnover  equivalency,  we  can  see  In 
some  Instances  dramatic  variation  as  with  the  Rochester  to  GE  Burlington  offices  greater 
than  three  to  one. 

The  line  Item  and  unliquidated  percent  distribution  present  a rather  different 
picture  as  you  can  readily  see.  Raytheon  and  GE  Burlington  stand  out  as  high  value 
locations  or  complex  types  of  product. 


SLIDE  119  ON 
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The  next  slide  adds  to  the  percent  characteristics  the  operational  nanpoMer  by 
location. 

I draw  your  attention  to  the  "authorized  positions"  colwn  and  the  "adjust  for  Ttf** 
colunn.  Currently,  we  have  United  our  transportation  and  packaging  operating  divisions 
to  four,  however,  these  four  elenents  provide  transportation  and  packaging  services  and 
support  throughout  the  region.  In  effect  they  serve  all  twelve  operational  locations. 

This  anonaly  required  an  adjustnent  to  the  actual  manpower  authorized,  the  last  colian. 
to  prorate  the  man  year  effort  of  the  servicing  office  and  offices  being  served. 

One  can  readily  see  the  variation  by  characteristic,  office  Iqr  office,  but 
collectively  the  workload  characteristics  are  relatively  balanced. 
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u * 

On  tMO  relatively  recent  occasions,  significant  authorized  nanponer  reductions  were 
Mandated  and  were  acconpllshed  on  the  basis  of  this  analysis.  Me  were  Initially  faced 
with  sonewhat  Irate  operating  officials  ready  to  request  reassessaent.  Uhen,  however, 
this  rationale  and  methodology  was  used  and  explained,  the  line  operating  elcants 
accepted  the  retrenchment  decisions  as  being  equitable  and  reasonable!  I emphasize  that 

no  quantitiaole  formula  was  used  except  workload  patterns,  trends,  and  experienced 
Judgment.  Authorized  position  distributions  were  then  adjusted  based  om  the  results  of 
this  CQiprehensIve  data  assembly  and  analysis. 

Earlier  I mentioned  the  Defense  Logistics  Agency  Integrated  Ptanagement  Engineering 
System  more  comonly  referred  to  by  the  acronj^  "DIMES*.  This  system  conforms  to  DOO 
Directive  5010.15  which  establishes  a requirement  for  management  engineering  skills  In 
the  military  departments  and  defense  agencies.  Simply  stated,  OIPCS  uses  the  expertise 
of  trained  management  engineers  to  determine  how  long  It  should  take  (not  how  long  It 
might  take  or  how  long  It  use  to  take)  to  perform  tasks  that  need  to  be  done  and  to 
establish  standards  by  which  an  organ1zat1on*s  manpower  efficiency  can  be  measured,  and 
to  seek  out  more  efficient  ways  of  accomplishing  tasks.  Once  the  management  engineer 
has  done  his  job,  the  results  are  made  part  of  the  data  base  In  a management  Information 
system  whether  It  be  a sophisticated  automated  type  or  a stubby  pencil  type. 

The  coomon  responsibility  for  defense  managers  and  supervisors  who  are  responsible 
for  the  effective  and  economical  use  of  their  manpower  resources  is  to  measure  and 
Improve  productivity  and  make  proper  use  of  productivity  changes.  From  the  perspective 
of  an  operating  manager  during  the  evolutionary  development  of  DIMES  some  difficulties 
were  encountered. 
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In  the  early  stages  of  DIMES  ibiplementatlon,  "engineered”  standards  were  developed 
and  set  at  a particular  location  and  subsequently  applied  to  another  location.  One  work 
center  had  two  primary  points  of  workload  count:  contracts  received,  and  contracts 
completed  and  closed  out.  The  work  standard  for  contract  closures  was  greater  than 
contract  receipts.  As  an  operating  supervisor,  I found  myself  In  the  unenviable  situation 
of  realizing  an  increase  In  contracts,  more  receipts  than  closures,  with  an  Inefficient 
productive  hour  condition,  a sunaary  Indication  that  my  staff  was  too  large  when  actually 
the  reverse  was  true..  Needless  to  say,  the  standards  were  eventually  adjusted  to 
accooBodate  the  actual  workload  conditions. 
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Subsequently  engineered  standards  nere  developed  for  each  location.  Albeit  this 
Mas  an  1i^>roveaient  from  the  earlier  iKthodology  It  left  unexplained  why  standards 
differed  fron  location  to  location  for  apparently  similar  tasks.  Furthermore,  It  did 
not  allow  correlation  coiqparlsons,  particularly  from  the  national  viewpoint. 

I 

OLA  Is  currently  In  the  process  of  establishing  national  standards  which  would  be 
applicable  to  all  and  any  cost  center  of  a similar  nature.  This  study  Is  known  as  the 
special  purpose  data  or  SPO.  The  SPD  standards  are  using  optimistic,  pessimistic,  and 
most  likely  time  aspects  with  one  time  selected. 


4- 


It  Is  certainly  not  my  Intention  to  either  demean  any  current  efforts  smch  as  the 
special  purpose  data  study  or  the  people  who  are  working  so  assiduously  to  make  them  a 
success  but  conversely  to  contribute  to  their  success.  The  problem  I perceive  with  this 
method  Is  wide  deviations  from  the  standard  could  cause  apparent  non-productive  conclusions 
when  In  fact  there  were  valid  deflations  due  to  differing  workload  conditions  and  quanti- 
tative differences  of  tasks  which  were  not  measured  In  establishing  the  standards. 

If,  however,  engineering  standards  were  developed  on  some  of  the  abstract  measure- 
ments 1 have  discussed  today,  several  advantages.  In  the  author's  opinion  would  be 

First,  the  universal  work  measurement  standards  would  be  facilitated  because  they 
would  be  based  upon  criteria  which  would  be  associated  with  contracts  at  a base  line 
that  identifies  micro  work  elements  keyed  to  actual  demographic  distribution. 


) Secondly,  It  would  accaaK>date  different  workload  nix  situations  with  wide  deviations 
at  all  locations. 


Centralized  control  of  the  standards  could  be  maintained.  As  operational  methods 
change  necessitating  revisions  to  existing  standards  or  the  establishment  of  new 
standards,  the  central  focal  point  with  the  complete  statistical  universe  would  logi- 
cally make  these  determinations. 

Standards  established  on  the  characteristics  discussed  would  promote  the  use  of 
AOP  systems  as  the  product  of  the  standards  and  dimensions  divided  by  the  appropriate 
weighted  factor  equating  to  the  resource  needs.  The  constants  then  are  the  standards 
and  weighting  factor  and  the  variables  (workload  data)  are  computed  by  AOP.  A spin  off 
benefit  to  the  computer  use  would  be  : Prograimned  regression  analysis;  prograsnned  correla- 
tion of  current  and  historic  data;  and  seasonally  adjusted  forecast  workload  requirements. 

The  final  point,  as  I envisioned  as  our  ultimate  objective.  Is  that  reallocation 
j of  our  human  resources  could  be  accomplished  more  accurately  and  more  timely  as  shifts 
In  workload  occur  from  one  geographic  area  to  another  and  more  responsive  adjustments  as 
workload  Increases  or  decreases. 
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If,  In  fact,  there  Is  any  potential  application  of  the  'production  contract 
attribute  systoa*  and  If  DCASR  Boston  Is  a nlcrocosai  of  the  DLA  CAO  Mcrocosn,  then 
It  would  be  even  mart  appropriate  to  apply  these  concepts  to  the  larger  sUtlstIcal 
ccanunlty  for  nore  accurate  analysis  and  conclusions.  . 
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I oust  oMwisIze  that  the  current  systems  llniutlons  exist  aerely  to  wore  accurately 
^flne  workload  and  relate  authorized  manning  to  It.  • There  is  great  deal  more  to  the 
distribution  of  authorized  positions  to  locations  that  Is  required  suck  as:  position 
management,  position  classification,  organization  structure  and  behavior  Mich  1 have 
not  addressed  and  which  I do  not  mean  to  over  slBq>11fy  In  this  paper.  There  are  some 
tasks  which  are  not  associated  with  contracts  and  consequently  do  not  correlate  to  these 
characteristics.  For  exas^le.  Industrial  preparedness  planning.  Industrial  resources 
and  perhaps  pre-award  surveys,  cost/schedule  control  systems  criteria  programs,  should 
cost  and  prtwhictlon  readiness  reviews.  These  efforts  should  be  assessed  from  a progrma 
basis  and  data  that  Is  Indigenous  to  those  functions.  Most  of  the  resources  are  related  i 
to  these  dimensions  and  the  regional  staff  should  be  treated  as  a percent  to  operations  H 
similar  to  an  overhead  cost  of  doing  business.  (The  production  directorate  regional  It 
staff  Is  8.0X  29  to  362.)  Clerical  position  requirements  can  normally  be  determined 
by  a clerical  to  professional  staffing  pattern  or  ratio.  Nothing  of  course  Ukes  the  • * 
place  of  sound  management  Jud^Mnt  but  Justification  for  deviation  due  to  anomalies 
should  be  ascertained. 


As  perhaps  most  research  studies  conclude, 
to  Improve  and  refine  the  system. 


research  and  analysts  Is  needed 


Finally  unlike  your  newscast  editorials,  I assu 
paper,  the  opinions,  and  conclusions. 


total  responsibility  for  the 
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INTRODUCTION 

This  Innovative  system  Is  the  result  of  a task  assigned  within 
Department  of  Athqt  Materiel  Development  and  Readiness  Conmand.  OARCOM. 
to  consider  the  application  of  word  processing  to  achieve  greater 
standardization  of  the  procurement  business. i As  the  Inflation  spiral 
quietly  but  surely  erodes  th6  operating  budgets  of  both  government  and  'TT 

Industry,  sometimes  overshadowing  our  efforts  to  obtain  higher  efficiency, 
the  specter  of  ever  dwindling  resources  prompts  us  to  explore  even  more  ( ) 

efficient  and  economical  methods  of  accomplishing  our  mission.  Such  was  ' 

the  case  In  the  early  months  of  calendar  year  1975  - the  problem  of  ad- 
ministrative costs  of  procurement.  Inltlelly.  Automatic  Data  Processing  i ) 
(AOP)  was  reputed  to  be  the  manager's  solution  to  reduced  manpower.  In  ‘ 

actuality.  ADP  was  supplanted  for  manual  record  keeping  as  an  "after-the- 
fact"  application  of  procurement  awards.  In  no  Instance  was  ADP  provid- 
ing a totally  mechanized  award  except  for  the  Defense  Logistics  Agency  (DLA)  ' I 
program  on  small  purchases.  The  DLA  program  provides  total  automation  ~ 

but  Is  restricted  to  procurement  buys  that  are  for  cataloged  priced  Items 
under  $10,000.  The  OD  Form  1155  and  Standard  Form  36  are  utilized  to  ' / 

reflect  the  data  and  applicable  provisions  and  acceptance  required  by  the  pre- 
printed  OD  1155-R.  The  computer  operated  system  records  the  data  and  produces 
management  reports.  However,  the  restriction  to  $10,000  could  not  be  i 

overcome  without  complete  redesign  and  programming  effort  to  accomplish 
tJie  requirement  to  automatically  produce  all  Instruments  regardless  of 
dollar  value.  Rather  than  adopt  this  system  as  the  answer  to  small  pur- 
chases, we  determined  to  pursue  a development  that  would  encompass  all 
Instruments  within  a systems  approach,  rather  than  Interconnect  various 
sub-systems.  Some  semi -automated  programs  currently  produce  data,  but 
In  no  case  provide  the  ASPR  clauses  and  references.  This  portion 
of  the  procurement  Instrument  was  being  produced  by  attaching  "boilerplate" 
pages  of  clauses  and  narratives.  Word  Processing  - the  application  of 
systems  analysis  and  electronic  technology  In  pr^uctlon  of  typed  words  - 
Is  a conceptual  cousin  of  ADP.  The  use  of  sophisticated  mechanization 
provides  more  typed  material  with  the  same  labor  input.  Word  processing 
provides  a dependable  supply  of  typing  assistance  to  management  and 
affords  a new  career  path  in  learning  new  techniques  and  developing  new 
skills  to  the  typist. 2 
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In  early  1975  a review  of  word  processing  as  applied  to  procurement 
operations  was  made  on  how  procurement  assignments  had  been  optimized. 

In  one  Instance,  a word  processing  center  had  been  established  as  a 
dedicated  group,  responsible  for  processing  and  capturing  contractual 
data.  Standard  clauses  and  other  pertinent,  repetitive  contract  data 
were  placed  on  magnetic  cards  and  systematically  inserted  into  so1ic1> 
tations  or  contracts  at  the  time  of  their  preparation.  The  advantages 
were  increased  productivity,  monetary  savings,  and  efficiency  as  well 
as  standardization  of  the  procurement  applications.  At  this  time  the 
task  became  a joint  effort  with  Directorate,  Management  Information  Sys- 
tems, to  capitalize  on  word  processing  systems,  particularly  in  the 
contracting  aspects  of  the  Coomodity  Command  Standard  System  (rcSS). 

Some  of  the  questions  to  be  answered  were:  “ ^ 

1.  Are  we  using  word  processing  techniques  and  equipment  in  prepa- 
ration of  solicitations  and  contractual  awaits?  . ■ ' 

^ • ■ * !•  . ^ * > U. 

2.  Can  we  do  more?  • t-v  f- : -v'. 

3.  Do  we  have  a standard  system?  ' ' ^ j 

'j4.  ' Have  Instructions  in  this  area  been  disseminated? 

5.  Can  we  further  automate  purchasing  and  contracting  functions  by 
the  use  of  mini-computer  with  integrated  word  processing  equipment? 

Consideration  was  given  to  take  full  advantage  of  the  data  already 
in  the  Coomodity  Conmand  Standard  System  (CCSS)  computers  as  well  as  the 
automatic  update  of  contractual  data  to  the  CCSS  data  base. 3 

Decisive  action  was  taken  in  May  1975  by  the  Amy  to  examine  the  pro- 
blem at  its  very  roots  - the  word  processing  workload  in  the  procurement 
area. 4 A Task  Force  was  developed  at  the  Army  Missile  Materiel  Readiness 
Conmaiid  comprised  of  personnel  with  experience  and  expertise  in  Automatic 
Data  Processing  and  Functional  Procurement.  The  command  was  given 
the  functional  responsibility  for  system  design  in  October  1975.  Conf^ls 
were  identified  as  follows:^ 

1.  The  mini-computer  would  be  Installed  in  the  procurement  area. 

2.  The  Directorate  for  Management  Information  Systems  would  have 
control,  maintenance,  and  operation  of  the  mini  with  terminal  operators 
to  be  provided  by  procurement. 

Initially  assigned  projects  to  be  developed  and  progranmed  encompassed: 

1.  Solicitations  for  firm-fixed  price  hardware  over  $10,000. 

2.  Small  Purchase  system  to  incorporate  the  quotation  feature. 


366 


/ 


update  to  the  Materiel  Acquisition  and  Delivery  (MAD) 
Tnif  recording  purchase  requirement  data)  and  Hilita^  Standard 

tontraa  Administration  Procedures  (MILSCAP)  Master  Kile  of  tteltlidart^ 


Also  included  as  possible  follow-on  requir«ents  were: 
1»  Modifications  for  the  above. 

2.  00  350  preparation. 


f v-  y 


y»'-  ■ *0  f j 54 

• fii: 


the  CCSs!'°*“*^  "odifications  posted  to  the  MILSCAP  Master  File  (MMF)  of 

o:  • 


3 


of  a System  Concept  Document.  It  was  recognized  that  a totally  automata 
wntractual  insti^t  preparation,  utilizing  source  dat?f?oi^«iJ?iSj^ 
OCSS  records,  had  not  been  enconpassed  in  the  CCSS.  Therefore  the  entire 
requirewMits  for  automation  within  the  Procur«nt  and  plSSSio? frS  JTd 

rMii4.»aw4  ♦ of  CCSSy  that  fo1104-on  design  and  prograiminq  would  be 

r^uir^to  complete  the  functional  processing  JequireSeSsTSe^riJuJl 

Our  initial  analysis  revealed  that  the  word  processors  currently  in  u<e 

investment^^’cSIilili!'  **ould  optimize  benefits  from  the 

inves^t.  CunrMtly,  word  processors  used  in  the  procurement  area  are 

originating  from  a typewriter.  The  first  type  is  a 
key  board  that  creates  an  extract  tape  or  card  in  the  exact  format  as  the 
^ second  tj^  is  a key  board  that  requires  the  pro- 
Sr^Sfrli  ^ clocked  in  exact  character  length  in  order  to^rmit 

-S  ^ reading  Of  the  typed  document.  In  sane  cases  it  is  possible 

prepared.  Even  though  the  advantages  of  Increased 

can^be^accomDl'u!2?h^  savings,  and  efficiency  as  well  as  standardization 
^ processing,  the  then  present  method  of 

thA°2dIi!!!!!^"^«.'^*^  the  ulitimate.  The  decision  was  made  to  utilize 
the  mini-cmputer  as  a high  level  of  word  processors  to  produce  all 

K and  cJSsisUng  0?  a key 

equipment  determined^ 

as  a requirement  for  the  system  are: 
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1.  A Text  Processor  to  align  the  printed  words  and  format,  as 
necessary,  based  on  additions,  cM"^®s,  and  deletions  to  previously 
stored  data. 

2.  Discs  and  tapes  to  store  accessable  data.  > 

* - ' •a*,- 

3.  A high  speed  printer  to  be  attached  to  the  mini-computer  to 
produce  the  contract  instruments  for  manual  review  and  concurrence  prior 

to  printing  the  total  quantity  for  distribution.  ^ 

4.  A connunicatlons  link  connecting  the  procurement  location  with 
the  printing  facility  to  preclude  manual  distribution  of  documents  for 
reproduction. 

Even  though  the  word  processor  typewriters  would  not  be  used  for 
input  by  the  procurement  personnel,  they  could  interface  other  functi.yna1 
areas,  as  In  the  creation  of  the  detailed  scope  of  work  and  DD  Form  1423 
to  be  attached  to  the  outputs  of  the  mini -computer  to  encompass  the  total 
procurement  package  for  solicitation  or  award.  The  completed  systems  con- 
cept document  was  formulated  to  encompass  all  procurement  outputs  to  be 
geierated  from  a mini-computer  utilizing  terminal  input  In  a direct  access  - 
real  time  processing  environment-  ... 

The  system  concept  was  developed  to  produce  the  fcHowing  automated 
outputs: 

1.  All  solicitation  instruments  regardless  of  dollar  value  and 

contract  type.  . - 

2.  Amendments  to  solicitations. 

3.  Award  instruments  for  automated  and  non-automated  f.olicitations. 

4.  Contract  modification  instruments. 

5.  DO  350  and  related  documentation  preparation. 

6.  Selected  matrices  and  tables. 

7.  Simultaneous  update  to  Materiel  Acquisition  and  Delive'ir  (NAD) 
and  MILSCAP  Master  Files  of  COSS. 

The  development  and  implemeiitation  of  the  PAOD  System  will  provide 

for: 


1.  Control  and  management  within  the  Procurement  and  Prodi'ction  area 
of  the  data  generated  as  a result  of  solicitations,  contractual  awards, 
and  modifications  thereco. 

2.  Responsive  support  to  the  procurement  area  vrhich  will  in  turn 
provide  better  visibility  to  other  functional  areas,  e.g.,  logistics, 
research  and  development,  and  financial. 
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3.  Standdrdizatlofl  of  contract  proparatlon  and  data  a1lQnn!&.t  on 
the  prescribed  contractual  forms. 

4.  . Assurance  that  tne  documents  prepared  and  the  data  generated 
therefrom  for  CCSS  update  a-e  complete,  compatible  and  valid  prior  to 
signing  the  award. 

5.  Better  utilization  of  resources  lunpower  in  overall  procuremei 
assigned  missions. 

■ " r.  • • . . 

6.  Capability  to  provide  current  data,  in  a more  timely  manner.  1 

nanagenient  reports.  • ...i- 

7.  Elimination  of  costs  associated  ir*th  keypunching  card  Input 

required  by  the  » nrent  CCSS  system.  - „ • . 


9.  Capability,  via  computer  progranclnc  and  storagz,  of  providing 
Instant  updati  under  controlled  conditlvos  at  terminal  locations 
all  codes,  tables,  matrices,  remarks,  and  ASPR  clauses. 

10.  Reduction  of  data  elements  required  to  be  entered  manually  at 
time  of  solicitation,  award,  and  modification  therefc**  by  utilizing 
existing  data  previously  recorded  in  CCSS  and  the  automated  procuraent 


Instrument  mechanized  files. 


SCOPE  AND  PURPOSE 


The  name  of  Procurement  Automated  Data  and  Document  System  (PADOS) 
assigned  to  cover  the  following  modules  or  processes: 

1.  Solicitation  Instrument  - Automated  Process. 

2.  Contract  Instrument  - Automated  Process. 


3.  Hodif icatlc.i  Instrument  - Automated  Process. 

4.  Reject  and  Suspense  Process. 

5.  Management  Requirement  Process. 

The  basic  functions  to  be  performed  by  the  .roposed  systems  are: 

1.  S'*iicitation  Instrument  - Automated  Procc  s:  ^ 

a.  Basic 

The  initial  action  required  in  tho  process  of  creat  ig  an  automated 


solicitation  is  usally  the  selection  of  prospective  offer-'-^. 
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The  Standard  Automated  Bidders  List  (SABL^  has  been  developed  to 
provide  this  data  to  the  Contract  Specialist.  SABL  Is  not  an  integral 
part  of  PADDS  but  is  accessible  via  PAOOS  terminals.  When  SABL  is  to 
be  utilized  in  the  procurement  action,  the  solicitation  process  Is 
'Initiated  by  accessing  SABL  for  a list  of  prospective  offerors. 

Simultaneous  with  the  access  of  SABL,  PADDS  will  assign  and  reserve 
a solicitation  number  for  the  action  being  processed. 

When  the  presolicitation  actions  have  been  completed,  the  Contract 
Specialist  can  resume  processing  the  solicitation,  citing  the  reserved 
number.  In  order  to  key  all  actions  to  the  appropriate  number. 

The  decision  making  process  Is  performed  by  the  Contract  Specialist 
prior  to  the  resumption  of  processing  the  solicitation.  A brief  trans> 
cript  reflecting  those  decisions,  plus  the  selection  of  appropriate  codes. 
Is  furnished  to  the  terminal  operator  for  entry. 

Data  Is  arranged  and  entered  In  the  segments,  with  edit  and  valida- 
tion occurring  at  the  conclusion  of  each  segment.  Validation  Is  performed 
against  the  CCSS  data  base  and  PADDS.  In  the  event  data  Is  Invalid,  an 
In-the-clear  error  message  will  be  displayed  on  the  terminal  screen  for 
immediate  correction.  • 

Uhen  all  data  have  been  entered  and  validated  the  Contract  Specialist 
will  be  furnished  a hardcopy  of  the  solicitation  for  purposes  of  obtain- 
ing required  reviews  and  approvals.  When  all  concurrences  have  been  ob- 
tained and  the  Contract  Specialist  desires  to  Issue  the  solicitation,  an 
electronic  message  Is  transmitted  to  the  bulk  printing  plant  requesting 
the  appropriate  number  of  copies  and  mailing  labels.  This  Is  accomplish- 
ed via  a communications  link  between  the  procurement  area  and  Field 
Printing  Plant  over  which  PADDS  data  Is  transmitted  and  captured  on  disc 
for  reproduction.  This  feature  eliminates  the  need  for  hardcopy  trans- 
mittal between  these  two  processing  points. 

b.  Amendment 

Should  It  become  necessary  to  Issue  an  amendment  to  an  automated 
solicitation  the  Contract  Specialist  may  do  so  by  accessing  the  PADDS 
Solicitation  File,  keying  the  action  to  the  appropriate  solicitation 
number.  It  will  not  be  necessary  to  reenter  data  that  Is  pertinent  to 
the  amendment,  since  it  is  on  record  In  the  solicitation  file.  The 
only  data  to  be  entered  Is  that  which  Is  being  changed  or  that  which  Is 
being  added  or  deleted.  The  amendment  number  will  be  computer  assigned. 

Hardcopy  processing  will  be  accomplished  in  the  same  manner  as  the 
basic  solicitation. 

2.  Contract  Instrument  - Automated  Process: 

a.  Award  Based  Upon  Automated  Solicitation 


Generation  of  a contract  instrument  to  make  an  award  resultino  from 

Soli^itTtiM  ® abbreviated  process.  The  PADDS 

File  will  be  accessed  to  obtain  the  majority  of  the  data 
^^red  for  generating  the  award  document  after  which  it  is  necess^y 
5 has  resulted  from  negotiations,  in  the  case 

adv^ised^  P’nKurement.  or  the  evaluation  process,  if  formally 


applied  to  the  generation  of  the  contract 
£52?^^  115“?  **  solicitation  process. 

"»ss»9es  advising  the  tenainal 
^ enter^.  Corrections  are  made  and  data  are 
1^1  Ida  ted  at  the  terminal.  Although  PADDS  is  not  desigiated  to 
pmess  fu^ng  documents,  it  will  effect  an  Interface  with  CCSS 
mI6S»  dt  th6  t1in0  dwdrd  Is  cont6nf>1dted9  for  tho  puroosc  of  notifvina 

Following  completion  of  the  entry  and  validation  of  data  elanents 
^e  aini-conqxiter  will  furnish  a hardcopy  for  the  purpose  of  obtaining 
approvals.  Accompanying  the  contractual  Instrument  will 

Procurement  Action  Report"  if 
required.  All  data  associated  with  the  contemplated  award  will  be 
retained  in  the  PADDS  Suspense  File  pending  execution  of  the  contract. 

b.  Awards  Based  Upon  Non-Automated  Solicitations 

S®  process  for  generating  a contractual  document  by 
^ ••esults  from  a non-automated  solicitation,  including 
tl2  instance,  there  are  no  pertinent  data  in 

tte  PADDS  Solicitation  File.  Consequently,  the  majority  of  data  elements 

^’'1?®®!®"*?®^  ^ solicitation  process  must  be 

Introduced  at  this  time  to  produce  the  contract  instrument.  In  addition 
taereto,  other  data  elements  associated  with  the  award  process  must  be 
inserted.  A first  impression  is  that  additional  effort  is  required  to 
iiake  an  awaM  when  the  solicitation  was  non-automated;  however,  under 
^oser  scrutiny  it  becomes  evident  that  the  above  assumption  is  not  true. 

0^  ‘lata  elements  of  both  phases  are 

combined  in  one  operation. 

Edit  and  validation  are  performed  as  in  the  previously  explained 
^CMS®s  and  a hardcopy  of  the  contract  instrument  and  accompanying 
OD  350  ar®  furnished  to  the  Contract  Specialist.  All  data  associat^ 
with  the  proposed  award  are  suspended  in  the  PADDS  Suspense  File  pending 
execution  of  the  contract. 


c.  Updating  Data  Bases 

Capturing  and  processing  data  resulting  from  awards  for  the 
^rpose  of  establishing  and  maintaining  a data  base  of  procuranent 
Information,  have  long  been  voluminous,  costly,  time-consuming 
Tnsks.  Whether  the  method  be  manual  or  automated,  the  consequences 
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remain  basically  unchanged.  PAODS  provides  update  of  both  the  CCSS  and 
Inherent  data  bases.  Little  or  no  additional  effort  Is  expended  In 
accomplishing  the  update  function  due  to  the  basic  concept  of  PAODS,  more 
specifically  defined  as  source  data  and  document  automation.  The  underlying 
principle  of  Source  Document  Automation  Is  the  practice  of  coding  select^ 
portions  of  the  data  appearing  In  procurement  Instruments  In  order  to 
capture  that  data  without  further  effort.  This  represents  a substantial 
advancement  over  manual  abstracting  and  optical  character  recognition  or 
magnetic  card  or  tape  methods. 

Release  of  the  captured  data  to  the  CCSS  data  base  updating  process 
Is  triggered  by  the  Insertion  of  the  Effective  Date  of  Aierd.  This 
action  also  serves  to  update  PADDS  files  In  which  award  Information  Is 
stored. 


3.  Contract  Modifications  - Automated  Process:  ' 

The  task  of  producing  a contract  modification  has  been  greatly  sim- 
plified by  PADDS.  The  system  accesses  the  PADDS  and  CCSS  files  to 
retrieve  data  accumulated  from  the  award  process.  To  acquire  data  from 
any  other  portion  of  the  contract  the  Conb'act  Specialist  need  only 
enter  the  contract  number  and  advise  the  system  of  action  contenplated. 
Data  elements  such  as  contractor  name,  address,  all  codes  and  other 
"header  data"  are  retrieved  and  prlnt^  without  further  effort  on  the 
part  of  the  Contract  Specialist.  The  modification  number  Is  assigned 
by  the  mini-computer  In  numerical  sequence. 

Provisions  have  been  made  within  the  system  to  accoaiodate  the  entry 
of  narrative  at  the  terminal.  This  makes  It  possible  for  the  Contract 
Specialist  to  tailor  the  language  of  the  modification  to  the  exact  terms 
desired.  This  feature  Is  also  applicable  to  all  phases  of  the  automated 
process. 

When  all  data  have  been  entered  and  validated,  a hardcopy  Is  produced 
within  the  functional  procurement  area  In  order  that  reviews  and  approvals 
may  be  obtained.  PADDS  Suspense  File  holds  the  action  pending  execution. 
Even  though  modifications  are  produced  in  numerical  sequence,  PADDS  will 
accept  the  notification  of  execution  of  modifications  that  are  out  of 
sequential  order. 

The  process  that  updates  the  data  bases  and  produces  appropriate 
reports  resulting  from  contract  modifications  Is  Identical  to  that  for 
basic  contract  awards. 

4.  Reject  and  Suspense  Process: 

Haxlmun  edit  and  validation  will  occur  at  the  time  the  data  Is  entered 
at  the  terminal.  In-the-clear  error  messages  appearing  on  the  terminal 
screen  will  tend  to  reduce  rejections  to  a minimum.  There  are,  however, 
some  Instances  where  final  validation  cannot  be  effected  at  the  terminal. 
The  Reject  and  Suspense  Process  provides  the  means  of  notifying  the  user 
of  Invalid  entries  discovered  In  final  validation,  and.  In  addition, 
thereto,  provides  for  retention  of  the  Input  pending  correction  of  the 
data.  When  the  errors  are  corrected  processing  will  continue. 


If  the  user  desires  to  Interrupt  processing  of  a procurement 
Instrument  for  reasons  such  as  notification  by  the  requiring  element 
.that  changes  to  the  requirement  are  contemplated,  PADDS  irlll  retain 
the  partially  completed  action  In  the  Reject  and  Suspense  File  for 
seven  working  days,  allowing  resumption  of  the  process  within  that 
period  of  time  without  loss  of  Input.  ... 

Solicitations,  Awards,  and  other  procurement  Instruments  that 
have  been  suspended  In  the  file  pending  Issuance  or  execution  will 
be  retained  until  disposition  Instructions  are  entered.  Reports  of 
suspended  actions  will  be  produced  upon  request  for  the  purpose  of 
Informing  management  of  their  status. 

5.  Management  Requirements  - Management  and  Requlnnent  Process: 

The  data  tase  that  will  be  established  as  a result  of  the  foregoing 
processes  will  constitute  a valuable  source  of  Information  from  which 
nna^ient  my  format  for  reporting  purposes.  The  Reject  and  Suspense 
File  contains  Information  pertinent  to  unawarded  actions.  The  data 
® result  of  awards  arc  pertinent  to  reporting  against 
total  mission  accomplishment.  Incidental  to  these  factors,  status 
reports  of  Individual  actions  may  be  extracted  from  PAODS  files. 

Reports  required  to  be  made  on  Individual  actions  have  been  ex- 
plalned  within  their  respective  processes.  The  contract  distribution 
list  and  mailing  labels  are  other  functional  type  by-products  of  PAOOS 
that  are  associated  with  the  processing  of  Individual  actions. 

Registers  listing  solicitations,  awards,  OD  350  reports,  and  a list 
of  procurement  actions  that  were  determined  to  be  exempt  from  pre- 
solicitation  synopsis  In  the  Connerce  Business  Dally  are  examples  of 
by-products  of  data  accumulation  In  PADDS.  The  full  extent  of  reporting 
requirements  cannot  be  accurately  determined  due  to  the  ever  changing 
circumstances  fac1;.g  the  procurement  world.  Offsetting  this  uncertainty, 
however,  is  the  capability  with  this  system  to  retrieve,  by  random  access 
any  data  contained  therein.  This  method  does  not  require  special  coding 
or  special  prograimiing  to  render  the  stored  data  accessible.  The  nature 
of  the  syste’i,  i.e.,  coin)os1t1on  of  documents  utilizing  coded  "building 
modules",  complements  the  random  access  feature  of  the  minicomputer.  It 
is  anticipated  that  the  combination  of  these  factors  will  produce  above 
the  average  capability  to  respond  to  the  multifaceted  reporting  needs  of 
the  procurement  area. 


DISCUSSION  OF  DETAILED  FUNCTIONAL  SYSTEMS  REQUIREMENT 


Identification  of  data  elements  relative  to  all  outputs  was  made  and 
the  elements  were  compiled  Into  categories. y The  first  category  contained 
elements  that  were  available  In  the  CCSS  data  base,  (MAO  File),  as  a 
result  of  a valid  procurement  requirement.  Subsequent  categories  defined 
the  elements  required  for  solicitation  and  attard,  HILSCAP  codes  and  com- 
ponents, data  for  the  DO  350,  distribution  list,  and  other  readily  Iden- 
tified management  reports.  Data  elements  Identified  by  two  or  more  names 
were  equated  for  systems  design  and  programing. 

Once  the  data  elements  were  Identified,  format  of  the  outputs  was  the 
next  consideration.  A comparison  of  the  data  elements  contained  on  the 
Standard  Form  36,  Continuation  Sheet,  currently  produced  by  the  Command's 
semi -automated  contract  instrument  system  was  made.  Requirments  of  ASPR 
2-201(a)  and  3-501 (b)(3)  were  also  reviewed.  As  a result,  the  determina- 
tion to  consolidate  all  data  elements  required  by  the  ASPR  Unlfonn  Contract 
Format  Sections  B,  C,  D,  E,  and  F was  made.  This  application  aligns  the 
supplies/ services  and  prices;  description/specifications;  preservation/ 
packaging/packing;  Insp^tlon  and  acceptance;  and  deliveries  and  perfor- 
mance data  Into  an  "entity”  of  data  to  be  contained  under  the  assigned 
Contract  Line  Item  Number  (CLIN)  Identification  on  the  solicitation  and 
award  outputs.  This  provides  the  viewer  a full  picture  of  the  requirement, 
stated  In  one  location,  rather  than  review  of  five  sections  to  ascertain 
the  total  essentials.  Special  consideration  was  given  to  the  reduction 
of  pages  saved  on  competitive  awards  due  to  the  austere  restrictions  placed 
on  paper  acquisition  and  useage.  The  decision  to  rearrange  the  sections 
of  the  Uniform  Contract  Format  (UCF),  required  an  ASPR  deviation  for 
authority. 3 The  deviation,  which  affects  both  the  Table  of  Contents  and 
Uniform  Contract  Format,  and  pertains  to  both  formal  and  negotiated  procure- 
ments will: 

1.  Realign  all  the  elements  required  by  the  Uniform  Contract  Format 
In  such  a manner  that  will  alleviate  the  problems  encountered  In  current 
efforts  to  generate  a contractual  document  containing  only  the  data  per- 
tinent to  the  awarded  Instrument  by  merely  removing  the  pre-award  data 
pages  in  a random  manner.  The  current  method  requires  restatement  In  the 
"List  of  Documents"  (Section  M)  as  to  the  pages  that  have  been  withdrawn 
and/or  renumbered,  causing  considerable  confusion  and  often  raising  doubts 
as  to  what  actually  constitutes  the  Intended  document. 

2.  Separate  pre-award  data,  e.g.,  representations,  certifications, 
and  other  statements  of  the  offeror;  solicitation  Instructions,  conditions, 
and  notices  to  offeror;  and  evaluation  factors  for  award.  This  information 
would  no  longer  be  co-roingled  with  data  required  for  post  award  functional 
applications  (administration,  production  management,  inspection,  acceptance, 
payment,  closeout,  and  retirement). 
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3.  Result  In  an  ancillary  cost  savings  from  the  realignment  as  the 
pages  required  for  pre-award  data  would  not  have  to  be  reproduced  and 
perpetuated  in  the  awarded  instrument.  In  some  forms  of  contracting 
this  reduction  constitutes  as  much  as  45X  savings  in  the  number  of 
printed  pages  required.  ^ , 


Statistics  compiled  for  a DoO  wide  application  are  sumnarized  in 
support  of  the  cost  savings:,  . - o • v- v-  L*,: 
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, . r CURRENT  PROCEDURE 


: '•r.  tr  rt.*'. 

Total  Pages  Printed  . , 

No.  of  Pages  Saved 
Per  Award  . , t 

Total  No.  Saved  Per 
DoO  Awards  of  39,436 
Per  Annum  . 


COMPETITIVE 


PROPOSED  PROCEDURE 
COMPETITIVE  REARRANGED 


SOLICITATION  6 AKARD  UNIFORM  CONTRACT  FORMAT 


'5.443 
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63.610.268 


CURRENT  PROCEDURE  PROPOSED  PROCEDURE 

NONCOMPETITIVE  NONCOMPETITIVE  REARRANGED 

SOLICITATION  & AWARD  UNIFORM  CONTRACT  FORMAT 


Total  Pages  Printed 

No.  of  Pages  Saved 
Per  Award 

Total  Number  Saved  Per 
DoD  Awards  of  59,056 
Per  Annum 


1,222 
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SUWARY  OF  COMPETITIVE 
AND  NONCOMPETITIVE  SAVINGS 

104,358,908 


In  addition  to  the  paper  savings,  collateral  costs  per  printed  page 
could  be  determined  by  each  agency.  Converting  the  savings  to  a minimum 
monetary  value  at  one  penny  per  page,  the  estimated  amount  exceeds  one 
million  dollars  annually  plus  attendant  costs  of  postage. 
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4.  Consolidate  Table  of  Contents,  Part  I,  Sections  B through  F 
(proposed  deviation),  e.g.,  supplies/services  and  prices;  description/ 
specifications;  preservations/packaging/packing;  inspection  and 
acceptance;  and  deliveries  and  performance  data  under  the  Contract  Line 
Item  Number  (CLIN)  on  the  SF  36  Continuation  Sheet.  However,  it  must 
be  recognized  that  additional  sheets  containing  special  explanatory 
provisions  for  inspection  and  acceptance  and/or  deliveries  and  perfor- 
mance may  be  required  and  these  provisions  have  been  encompassed  in  the 
deviation. 

5.  Benefit  all  parties  concerned  with  a post  award  document,  in 

reading  data  pertinent  to  a particular  Contract  Line  Item  Number  (aiN), 
without  the  usual  search  through  numerous  pages  in  order  to  locate  ad- 
ministrative data  relative  to  any  specific  CLIN.  - 

The  benefits,'  as  described  above,  are  of  a stand-alone  nature. 

However,  when  coupled  with  the  mini-computer  application,  other  benefits 
are  to  be  realized.  Some  of  these  are:  . ^ i 

1.  The  incidence  of  errors  in  manual  preparation  would  be  ’ 

greatly  reduced  by  automation  as  a result  of  extracting  pre-validated 
data  from  the  mainframe  computer  and  by  recall  of  stored  data  in  the 
mini-computer.  j ■ 

. . . * 5 ■ ^ ^ - '-“S"  ' 

2.  The  Instantaneous  display  of  data  Input  prior  to  creating  the 

contractual  document.  This  fact»  in  itself,  will  reduce  administrative 
modifications.  * 

Coupled  with  the  decision  to  realign  the  Table  of  Contents  and  Uniform 
Contract  Format  was  the  determination  to  let  the  mini-computer  "free-form" 
the  data  to  be  printed  on  the  SF  36.  The  mini  will  print  headings  above 
the  variable  data  thus  allowing  for  fluctuating  narrative  and  instructions 
to  be  printed  for  each  line  item.  This  will  also  provide  the  capability 
for  continuous  printing  without  changing  forms.  The  forms  comparable  to 
the  SF  26,  SF  30,  and  SF  33,  granted  under  the  deviation  for  conducting  a 
Source  Data  Automation  (SDA)  test  for  word  processing  (typewriter  or  optical 
character  reader),  were  reviewed.  Even  though  the  PADDS  documents  will  be 
computer  generated  and  not  require  specified  data  element  limitations,  it 
was  decided  to  utilize  the  SDA  forms  for  the  cover  or  front  page  of  the 
contract  instruments  for  the  sake  of  standardization.  ' ’ 

In  addition  to  providing  for  the  complete  procurement  business  under  the 
automated  PADO  System,  a number  of  innovative  management  tools  were  also 
developed. 


The  Regulation  Reference  Record  was  developed  to  store  all  references/ 
clauses/narratives  required  by  ASPR,  Army,  DARCOH,  and  local  Major  Sub- 
ordinate Command  (MSC)  to  be  produced  on  the  contractual  instruments  by  the 
computer.  A six  position  element  code  was  appended  to  each  clause/reference 
in  the  file,  and  by  the  entry  of  this  code,  the  complete  title,  date,  and 
narrative  is  able  to  be  generated.  The  first  digit  of  the  element  code 
identifies  the  section  of  the  ASPR  Uniform  Contract  Format  in  which  the  item 
appears  if  the  UCF  is  applicable.  The  second  digit  identifies  the  authority 
or  regulatory  requirement: 
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A - Armed  Services  Procurement  Regulation  (ASPR) 

D - Department  Regulation  (Army) 

M - Major  CoBinand  Regulation  (DARCON)  - r 

S - Major  Subordinate  Conrand  Regulation  (MIRCOM) 

I - Commodity  Comnand  Standard  System  (CCSS)  data  elonent 

The  third  position  identifies  whether  the  clause  is  to  be  stated  as 
full  text  or  by  reference  only.  The  final  three  digits  are  numeric,  com- 
mencing with  001  for  each  section  of  the  UCF  and  generally  will  be  in 
numerical  sequence  throughout  each  section.  To  preclude  duplicate  items 
from  having  to  be  loaded  to  the  record,  as  for  data  elements  appearing  on 
automated  solicitations  ana  awards  utilizing  the  DO  Form  1155.  the  first 
two  ^sitions  are  coded  "05"  to  identify  that  sections  of  the  UCF  are  not 
applicable.  When  used  in  automated  small  purchases,  article  numbers  will 
be  assigned  to  the  elements  as  they  print  out  on  the  instrument.  Based 
on  the  dollar  value,  type  of  solicitation/award  code,  type  of  line  item 
being  procured,  and  special  cost  code  the  computer  will  automatically  ex- 
tract the  required  standard  regulation  references  applicable  to  the  specific 
award.  Ability  to  add.  change,  or  delete  individual  references  is  provided 
on  an  "as  required"  basis  by  the  use  of  the  element  code  described  above. 

Such  changes  will  not  alter  the  data  base  but  will  allow  for  “tailoring" 
individual  requirement;.  An  errata  sheet  will  be  produced  with  each  con- 
tractual instrument  and  where  changes  have  been  made  to  the  standard  nar- 
tativfc,  will  identify  the  specific  data  element  and  page  number  on  which 
the  change  is  printed.  Where  no  deviations  occur  the  notation,  "No  deviations/ 

changes  are  included  in  solicitation/contract  number ■ 

will  be  printed.  This  application  will  optimize  the  manual  review,  and 
upon  completion  of  required  review  will  be  filed  in  the  pre-award  portion 
of  the  procurement  package.  The  properly  annotated  errata  sheet  will  be 
maintained  to  show  that  each  reviewing  office  having  cognizance  over  the 
contents  of  the  contractual  instrument  had  knowledge  of  the  changes.  The 
production  of  only  applicable  clauses,  and  references  will  alleviate  the 
current  practice  of  "boilerplating"  and  then  stamping  "Delete"  over  the 
references  that  are  not  applicable  to  the  individual  solicitation/award. 

This  record,  as  well  as  the  other  matrices  may  be  updated  at  the  terminal, 
but  will  be  under  a controlled  environment  to  assure  compliance  and  correct- 
ness of  data. 

The  Weapon  System  Matrix  was  developed  to  automatically  produce  the 
Army  system  or  equipment  code  required  for  DO  350  preparation.  The  matrix 
is  controlled  by  the  Weapon  System  Code  and  cross-references  to  provide 
the  Army  Budget  Activity  Code  (Command  Management  Structure  Code);  Federal 
Supply  Class  (FSC)  or  Service  Code;  System  or  Equipment  Code;  and  the  De- 
scription of  Coimiodity  or  Service.  The  matrix  will  produce  the  In-the-Clear 
data  as  well  as  the  codes  on  the  Individual  Procurement  Action  Report,  The 
description  for  supplies  and  equipment  will  be  computer  generated  from  the 
command  management  structure  code  contained  in  the  MAD  File.  As  in  the  other 
matrices,  an  override  capability  exists  in  this  matrix  also,  to  further 
define  the  description  when  warranted. 


L)  The  Buyer  Identification  Matrix  contains  the  two-position  code  assigned 


to  each  Contract  Specialist  within  the  comnand.  The  entry  of  the  Buyer  Code 
retrieves:  Buyer  Name,  Including  title,  first  and  last  naM,  Attention 
Symbol  of  the  Buyer's  Procurement  Office;  Telephone  Extension,  Including 
Area  Code,  Commercial  and  AUTOVON  Number;  Depository  Location,  Room  and 
Building  Location  where  Buyer  resides;  Coamand  Identity  and  Geographical 
Location/City,  State  and  Zip  Code/  This  matrix  will  be  the  responsibility 
of  each  command,  with  update  via  terminal  entry. 


The  Contract  Payment  Notice  Recipient  Matrix  is  a cross-reference  from 
the  Fiscal  Station  Nimiber  in  the  accounting  classification  to  the  appli- 
cable DoDAAO  Code  for  the  Contract  Payment  Notice  Recipient.  This  matrix 
will  produce  the  code  required  for  MILSCAP.  As  with  the  Regulation 
Reference  Record,  this  matrix  is  controlled  by  a central  point  within  the 
comnand  to  assure  updating  on  a current  basis  as  well  as  control  over  the 

ddtA.  ^ 'O 

The  Contractor  Identification  Header  Matrix  which  is  controlled  by 
the  Contractor  Identity  Code  from  the  H-8  handbook,  is  the  most  complex 
of  all  matrices.  The  Contractor  Identification  code  will  generate  the 
codes  and  addresses  for  the  Contract  Administration  Office  (CAO),  Contract 
Administration  ADP  Point,  and  Paying  Office  as  well  as  the  address  of  the 
contractor.  Where  a Contractor  Facility  Code  is  applicable,  this  entry 
will  generate  the  applicable  codes  for  the  DO  350  preparation.  If  a 
facility  is  non-appli cable,  the  DO  350  codes  will  be  generated  from  the 
Contractor  Identification  Code.  The  matrix  will  produce  all  of  the  codes 
and  related  addresses  for  contracts,  modifications,  and  DO  350  outputs 
and  will  print  the  applicable  data  in  the  prescribed  blocks  on  the  forms. 
Changes,  additions,  and  deletions  are  entered  from  the  terminal  to  update 
the  matrix.  This  matrix  is  also  controlled  by  a central  point  within  the 
comnand  to  assure  that  all  changes  are  incorporated  on  a timely  basis. 
Overlay  technique,  via  terminal  entry,  is  provided  to  enter  local  conmand 
codes  when  an  award  is  retained  by  the  comnand  for  administration  and 


payment. 


The  contractor's  zip  code  is  recorded  in  a separate  identifiable 
location  in  the  record  to  facilitate  inquiries  required  for  GAO  and 
congressional  requests.  This  matrix,  alone,  will  reduce  research  time 
and  provide  accurate  and  complete  data  to  be  generated  on  the  output, 
thus  reducing  administrative  modifications  to  correct  the  required  codes 
and  addresses.  The  matrix  will  be  manually  built  by  our  comnand  to 
meet  the  prototype  testing.  However,  action  is  being  initiated  by 
headquarters  to  acquire  a tape  update.  This  quarterly  update  tape  will 
be  converted  to  update  the  matrix  automatically.  Prior  to  this,  update 
will  be  accomplished  by  terminal  input. 

Other  management  improvements,  not  within  the  matrices  development 
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1.  The  Dollar  Threshold  Is  an  optional  code  which  may  be  entered 

when  there  is  reason  to  believe  that  the  anticipated  differences  of  all 
combined  Procurement/Work  Directives  (P/WD's)  is  within  25t  of  the  actual 
P/WD  totals.  This  entry  will  "override"  and  request  the  required  clauses 
and  narratives  for  the  next  higher  level  threshold  to  be  generated.  . , 

Seven  ranges,  from  $2,500  or  less  to  over  $1,000,000  are  identified.  ' 

2.  The  Contract  Line  Item  Number  Identity  Code  defines  the  CLIN 

requirement  and  recalls  the  appropriate  clause  definitions.  CLINs  are  , , 
identified  for  the  following  requirements:  . 

• V'  - ^ cTdNjt- 

a.  Production  Quantity.  , . 

^ i‘’  «.  ..i.,  ‘ 'orjci'i 

b.  First  Article  Quantity.  j,-};  .,i 

c.  Production  Quantity  Without  First. Article.  )Vt:  Krw  • 

4:1^-”;-.  A . ' ^ • !,  . J Y.:i  v»-«t  i 

d.  Option  Quantity.  c r?o  »^*J^5fc3v  5v 

e.  Multi-Year  Program  Quantity.  - ~ 


I ’ 

1 • ( 


\ J iVjt  i I ;r 

f.  Service  Line  Item. 


•U 


9* 

h. 

i. 

J. 


•Data  Item  - DO  250  not  required. 
Data  Item  - DO  250  required. 
Prototype  (Shop-Queen)  Item. 


Maintenance  and  Overhall  (Rebuild)  Item. 


r 


«T:i 
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3.  The  assignment  of  codes  to  delivery  schedule  data  has  optimized 
the  changing  or  revising  of  dates,  consignees,  and  HILSTRIP's  under  PADDS 
processing.  Instead  of  entering  the  existing  data  elements  requiring 
modification,  a Document  Nixnber  Relationship  Code  and  Delivery  Schedule 
Relationship  Code  were  developed  to  specifically  identify  the  line  or 
lines  to  be  updated.  The  use  of  these  codes  precludes  the  entry  of 
approximately  thirty  characters  to  specifically  distinguish  which  MILSTRIP 
line  is  affected.  When  an  item  is  required  to  be  diverted  from  one  consignee 
to  another  or  schedule  delivery  date  is  revised,  only  the  relationship 
codes  and  changed  data  need  be  entered  to  effect  the  modification  and 
generation  of  MILSCAP  abstract  transactions. 

4.  Awards  which  contain  options  may  cite  a range  quantity  applicable 
to  the  exercise  of  an  option.  In  these  cases,  a firm  Purchase  Unit  Price 
may  be  stated  or  a not-to-exceed  Purchase  Unit  Price  may  be  stated.  The 
option  CLIN  will  be  stated  as  awarded  with  the  Purchase  Unit  Price  also 
stated  but  no  quantity  and  no  extension  to  Total  CLIN  Amount.  Awards  which 
cite  single  quantity  options,  will  state  quantity.  Purchase  Unit  Price,  and 
Total  CLIN  Amount.  However,  this  amount  will  not  be  included  in  the  Total 
Amount  of  Contract  until  the  option  is  exercised,  even  though  the  CLIN 
amount  is  stated  on  the  hardcopy  instrument.  Data  required  to  be  printed 
on  the  contract  or  contract  modification  for  option  CLIN's  are  determined 
by  entry  of  the  Procurement  Request  Order  Number  (PRON)  for  the  initial 
awarded  identical  item.  This  entry  extracts  the  NSN,  FSCM,  Long 
Manufacturer's  Part  Number,  and  Noun  from  the  file.  When  an  option  is 
exercised,  a PRON/PRON  Amendment  will  be  generated  to  furnish  funding  and 
additional  data  elements. 
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5.  Awards  which  contain  incremental  funding  are  awarded  for  an  amount 
greater  than  initially  funded.  The  total  agreed  upon  contract  amount  will 
be  stated  on  the  hardcopy  instruments  as  well  as  the  total  obligated  amount 
by  accounting  classification  (ACRN)  and  the  Total  aiN  Amount  will  be  the 
sunnation  of  the  cost  of  all  phases  of  performance  Included  under  that 
CLIN.  In  the  case  of  multiple  CLIN's*  the  same  applies.  In  such  instances, 
the  sum  of  all  CLIN's  will  equal  the  Total  Amount  of  Contract.  However, 
for  NILSCAP  purposes,  the  obligated  and  Total  CLIN  Amounts  will  only  be 

for  the  initially  funded  dollars. 

6.  Awards  which  contain  multiyear  requirements  are  awarded  for  the 
total  of  all  program  years.  This  total  agreed  upon  contract  amount  will 
be  stated  cn  the  hardcopy  instrument.  Initially,  only  the  first  program 
year  will  be  funded  with  provisions  for  notification  to  the  contractor. 

on  or  before  a specific  date  or  event,  as  to  whether  the  succeeding  program 
year  will  be  funded.  This  procedure  continues  throughout  the  contract. 

In  the  event  that  subsequent  program  years  are  cancelled,  the  contract 
will  contain  the  formula  and  a limitation  by  whic.-  cancellation  costs  will 
be  computed.  The  hardcopy  instrument  will  contain  the  quantity.  Purchase 
Unit  Price,  and  Total  aiN  Amount  identified  to  a specific  aiN  for  first, 
second,  and  subsequent  years.  Options,  under  multi-year  requirements  will 
be  stated  as  defined  in  paragraph  4.  above.  Data  required  to  be  printed 
on  the  contract  or  contract  modification  for  subsequent  multi-year  require- 
ments is  determined  by  the  entry  of  the  PRON  for  the  initial  awarded  iden- 
tical item.  This  entry  extracts  the  NSN,  FSCM,  Long  Manufacturer's  Part 
Number,  and  Noun  from  the  file.  However,  for  NILSCAP  purposes  only  those 
aiN's  which  are  awarded  and  funded  for  the  first  program  year  will 
generate  NILSCAP  CLIN  transactions  on  the  basic  contract  instrument. 

The  above  option,  incremental  funding,  and  multi-year  applications  require 
that  the  Total  Amount  of  Contract  be  a manual  entry  in  lieu  of  the  com- 
puter generation  of  Total  Amount  of  Contract  from  the  sunmation  of  all  the 
Total  CLIN  Amounts.  The  CLIN's  assigned  for  options  and  multi-year 
requirements  will  be  retained  on  the  Contract  File  of  PAODS  for  future 
application. 


7.  A Delivery  Order  Header  Record  is  available  within  the  system  to 
benefit  the  Contract  Specialist  in  processing  orders  under  Delivery  Type 
and  Basic  Ordering  Agreement  awards.  Standard  data  elements  applicable 
to  the  basic  agreement  are  stored  for  automatic  retrieval . Only  the  basic 
agreement  number  need  be  entered  and  the  system  will  automatically  assign 
the  next  sequential  order  number  to  the  unobligated  requirement.  As  with 
other  requirements,  the  PRON  is  the  interface  to  generate  the  data  applicable 
to  the  CLIN  or  CLIN's.  Therefore,  this  application  provides  for  almost  total 
computer  generation  of  the  data  required  for  each  order. 

Extensive  logic  charts  were  developed  for  each  data  element  by  individual 
application.  Application,  input  or  output,  or  computer  generation  was 
identified;  interrelationship  with  associated  elements  was  documented;  and 
the  data  element  was  explained.  The  associated  Regulation  Reference  Code 
vas  appended  for  complete  visability  in  programming. 
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CONCLUSIONS 

The  system  design  and  development  have  been  considered  from  the  systems 
approach.  It  has  become  accepted  that  complex  systems  cannot  be  produced 
by  merely  Inter-connecting  component  sub-systems.  Any  operating  entity 
must  be  designed  from  an  overall  view  If  full  Integration  and  maximum 
effectiveness  are  to  be  achieved.  The  use  of  the  systems  approach  was  a 
determing  factor  in  Incorporating  all  procurement  applications  In  the 
PADO  System. ]Q 

Based  on  the  systems  approach,  as  well  as  the  other  unprecedented  applica- 
tions. the  provisions  and  benefits  of  the  PAOD  System  are  stated  below: 

1.  Utilize  source  data  and  source  document  applications  to  record 
data  at  source  of  entry  to  Improve  the  validity  and  timeliness  of  manage- 
ment reports  and  allow  the  contracting  officer  to  have  precise  control  over 
data  generated  from  contractual  Instruments.  „ . 


Z.  Reduction  of  data  elements  required  to  be  entered  manually  at  time 
of  solicitation,  award,  and  amendment,  or  modification  thereto,  by  utilizing 
source  data  previously  recorded  In  CCSS  and  PAODS  files. 

3.  Utilize  ASPR  clauses,  by  reference  and  In-the-Clear.  and  standard  • 
statements  In  the  generation  of  the  contractual  documents. 

4.  Assure  that  the  documents  prepared  and  the  data  generated  therefrom 

for  CCSS  update  are  complete,  compatible,  and  valid  prior  to  signing  the 
award.  . 

5.  Standardization  of  contract  preparation  and  data  alignment  on  the 

prescribed  contractual  forms.  , 

6.  Capability  to  provide  current  data  for  management  reports  In  a 
timely  manner,  thus  providing  better  utilization  of  manpowe;  and  better 
visibility  to  other  functional  areas.  I.e..  logistics,  research  and  develop- 
ment. and  financial . 

7.  Capability,  via  computer  programming  and  storage,  of  providing  o'* 
changing  at  terminal  locations  Immediate  update  to  codes,  tables,  matrices, 
remarks,  and  ASPR  references. 

8.  Ability  to  generate  only  tlie  specific  clauses  required  for  tailoring 
Individual  contractual  Instruments,  thus  obsoleting  all  “boilerplating”. 

9.  Ability  to  generate:  The  00-350;  solicitation  (SUN),  award  (PUN), 
and  00-350  registers;  Oellvery  Order  Header  Report;  distribution  list;  and 
other  management  reports  simultaneously  with  the  production  of  the  award 
Instruments. 
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10.  Ability  to  reduce  printing  and  paper  required  for  contractual 
preparation  as  a result  of  the  rearrangement  of  sections  of  the  ASPR 
Uniform  Contract  Format  and  to  allow  detachment  of  certain  sections 
required  for  solicitation  but  not  required  to  be  physically  furnished 
with  the  award  instrument.  . v.,  ^ 

The  PADO  System  milestones  project  prototype  testing  and  training  to  be 
conducted  by  the  coomand  in  late  1977  with  implementation  scheduled  for 
late  spring  1978.  , , i ii.  ^ 
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GLOSSARY 
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ACRN  - Accounting  Classification  Reference  Number.  • ’ ’ f .‘v 

• v..  /:  ' 

ADP  - Automatic  Data  Processing.  si  erw 

AUTOVON  - Automatic  Voice  Network.  ' “ ••  ••i.- 

' ' h;.'./ -f,-.  , ' • it  l5.v,>-> 

Boilerplate  - Bringing  together  standard  ASPR  clauses  normally  required  in 
the  majority  of  awards  and  printing  In  suquence  for  atta.hnent  to  the  other 
sections  of  the  award  Instrument. 

CAO  - Contract  Administration  Office. 

CAO  ADP  Point  - Contract  Administration  Automatic  Data  Processing  Point. 

CLIN  - Contract  Line  Item  Number  - (Used  to  encompass  Line  Item  Nunber, 
Contract  Sub-Line  Item  Number,  and  Exhibit  Line  Item  Number). 

CCSS  - Comnodlty  Coomand  Standard  System. 

DARCOM  - Department  of  Army  Hateriel  Development  and  Readiness  Comnand. 

DLA  - Defense  Logistics  Agency. 

DMIS  - Directorate  for  Management  Information  Sy-tems. 

DoO  - Department  of  Defense. 

DoDAAD  - Department  of  Defense  Activity  Address  Directory. 

FSCH  - Federal  Supply  Code  for  Manufacturers. 

GAO  - General  Accounting  Office. 

MAD  - Materiel  Acquisition  and  Delivery. 

MILSCAP  - Military  Standard  Contract  Administration  Procedures. 

HILSTRIP  - Military  Standard  Requisitioning,  Receipt,  and  Issue  System. 
m - MILSCAP  Master  File. 

KSC  - Major  Subordinate  Command. 

NSN  - National  Stock  Number. 
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PAODS  - Procuraient  Automated  Data  and  Document  System. 
PRON  - Procurement  Request  Order  Number. 


P/WD  - Procurement/Work  Directive. 


SABL  - Standard  Automated  Bidders  List.  . 


• SOA  - Source  Data  Aut(nat1on. 


UCF  - Uniform  Contract  Format. 
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FOOTNOTES 


1.  DARCOM  Spring  Coimanders  Conference  (1975),  SGS  Case  7708,  Subject 


Hord  Processing. 

2.  Word  Processing;  The  GSA  Finds  a Wav.  Gerald  Thacker  and  Robert 
Redtaend,  Govemnent  Executive,  Hay  1977. 


3.  Headquarters,  DARCOH  Manorandun,  dated  6 June  1975,  S " * 


Processing. 


4.  Headquarters,  DARCOH  Hemorandua,  dated  29  Hay  1975,  Subject:  Word 
Processing. 


5.  Headquarters,  DARCOH  Letter,  dated  8 October  1975,  Subject:  Computer 
Support  for  Procurement  and  Production  Directorate. 


6.  Headquarters,  DARCOH  Letter,  dated  19  November  1975,  Subject:  Syston 
Concept  Document  for  Hini-Computer  Support  of  the  P&P  Area  of  HSC's. 


7.  Headquarters,  DARCOH  Letter,  dated  7 July  1976,  Subject:  Detailed 
Functional  System  Requiremant  (DFSR)  - Procurement  Automated  Data  and 
Document  System  (PAOOS). 


8.  Headquarters,  HIRCOM  Letter,  dated  19  December  1975,  Subject:  Request 
♦or  Deviation  - ASPR  2-201 (a)  and  3-501 (b)(3)  (Uniform  Contract  Format 
and  Doc  A Test). 

9.  DoD  Award  Statistics,  Fiscal  Year  1975,  Department  of  Army,  Comptroller. 

10.  The  Systems  Approach.  Thomas  D.  HcGrath,  publication  and  date  unknown. 
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QARCOM  Spring  Conmanders  Conference  (1975),  SGS  Case  7708,  Subject; 

Uord  Processing. 

DoD  Award  Statistics,  Fiscal  Year  1975,  Department  of  Army,  Comptroller. 


Headquarters,  DARCOM  Letter,  dated  7 July  1976,  Subject:  Detailed 
Functional  System  Requirebient  (DFSR)  > Procurement  Automated  Data  and 
Document  System  (PADDS).  . * 4 ; ^ ^ 
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Headquarters,  DARCOM  Letter,  dated  8 October  1975,  Subject:  Computer  . >•';  ‘ '1 
Support  for  Procurement  and  Production  Directorate.  ' 

Headquarters,  DARCOM  Letter,  dated  19  NovenAier  1975,  Subject:  System  ' j 
Concept  Document  for  Mini-Computer  Support  of  the  P4P  Area  of  MSC's. 

Headquarters,  DARCOM  Memorandimi,  dated  29  May  1975,  Subject:  Uord  . 
F'^cesslng.  ^ , ..  .. 

Headquarters,  DARCOM  Memorandimi,  dated  6 June  1975,  Subject:  Uord  " . J 

Processing.  ^ 


Headquarters,  MIRCOM  Letter,  dated  19  December  1975,  Subject:  Request 
for  Deviation  - ASPR  2-201(a)  and  3-501(b)(3)  (Uniform  Contract  Format 
and  Doc  A Test).  ' ^ • 

The  Systems  Approach.  Thomas  0.  McGrath,  publication  and  date  unknown. 
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Mbrd  Processing:  The  GSA  Finds  a Uay.  Gerald  Thacker  and  Robert  Redmond, 
tevernment  Executive,  May  1977.  , . - j 
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Georganw  Roberts 

Oklahoma  City  Air  Logistics  Center 
INTROOUaiON 


In  the  Spring  1974  voIm  of  the  National  Contract  Nanagenent  Journal  Dale  McNabb 
discussed  a stotistically  oriented  quality  assurance  system  implemented  by  the 

*"<*  Production  at  Sacramento  Air  Ugistics  Center 
(SH-ALC),  Air  Force  Logistics  Command. • At  that  time  the  Directorate  of 
Procur^t  and  Production  at  Oklahoma  City  Air  Logistics  Center  fOC-ALC)  was 
searching  for  an  effective  means  of  measuring  the  quality  of  its  contractual 
actions.  Since  nianerous  indicators  were  available  to  measure  tiKliness  of 
processing  procurement  actions,  managers  could  easily  determine  where  delinquencies 
were  occurring  and  what  was  causing  them.  Unfortunately,  they  could  not  as 
readily  identify  quality  problems  and  pinpoint  their  source.  Too  often  they  ‘ 
relied  upon  external  criticisms  of  procurement  quality.  These  criticisms  were 
generated  by  the  Procurement  Conmittee  and  by  self-inspection  teams.  The 
Pi^urement  Committee  had  traditionally  been  responsible  for  contractual  qualUy. 
Thorough  reviews  of  contracts  over  $20O.D0O  were  performed  by  experienced 
analysts.  Not  so  thorough  reviews  of  contracts  under  $200 .ODD  were  performed 
by  a less  experienced  technician.  While  all  contracts  over  $200,000  were 
reviewed,  only  a very  small  sample  of  the  smaller  buys  were  scrutinized.  If 
some  area  of  the  procurement  operation  was  considered  deficient  by  the 
Inspector  General  (IG)  or  some  other  outside  critic,  self-inspection  teams  were 
established  to  examine  the  buying  divisions  to  determine  if  the  reported 
problems  were  present.  The  nunber  and  percent  of  contractual  actions  reviewed 
were  left  to  the  discretion  of  each  team;  i.e.,  scientific  sarpling  was  non- 
existent. Equally  uncoordinated  was  the  resolution  of  these  problems.  These 
*5  I??®  frequently  lacked  the  expertise  necessary  to  identify  the  causes 

of  IG  findings.  Therefore,  the  inmediate  supervisor  was  directed  to  make 
whatever  changes  he  considered  necessary  to  correct  deficiencies  in  his  own 
organization. 


Unfortunately  these  various  systems  of  quality  measurement  were  never  effectively 
coordinated;  i.e.,  no  one  gathered  all  of  the  findings  together  for  the  purpose 
of  identifying  trends  and  suggesting  corrective  action.  This  piecemeal  approach 
resulted  in  duplication  of  effort  and  an  incoherent  picture  of  procurement 
quality.  What  was  needed  was  an  integrated  quality  improvement  system  based 
on  statistical  sampling  procedures.  With  such  a system  managers  could  isolate 
the  areas  which  were  most  deficient  and  focus  their  attention  on  problems  that 
offered  the  greatest  potential  for  cure.  Instead  of  attacking  all  of  the 
problems  all  of  the  time,  they  could  concentrate  their  resources  in  the  areas 
offering  the  greatest  return  on  investment. 


> 

U 


.1 


The  Procurement  Integrated  Quality  Assurance  System  (PIQUAS)  used  at  SM-ALC 

appeared  to  be  a more  efficient  means  to  measure  quality.  The  advantages  of  «'  ' 

PIQUAS  were  its  statistical  sampling  techniques  and  review  sheets  made  up  of  ' 

objective  questions  which  could  be  answered  “yes"  or  "no“.  Disadvantages 
included  the  time-conswing  detailed  analyses  performed  on  each  discrepant 

file  and  the  small  number  of  contracts  included  in  each  month's  sample.  After  ! . \ ^ 

thorough  study  of  PIQUAS,  the  Director  at  OC-ALC  decided  to  adopt  a similar  — 
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quality  assurance  systea.  In  contrast  to  SM-ALC,  OC-ALC  planned  to  review 
the  largest  sample  possible  and  to  identify  trends  rather  than  perform  a 
detailed  analysis  of  each  file.  The  purpose  of  this  paper  is  to  illustrate 
Oklahoma  City  ALC's  adaptation  of  the  PIQUA  System  and  to  describe  the  results 
achieved. 

- r.. - 

DESIGNING  THE  PROGRAM  FOR  IMPROVEMENT  OF  CONTRACTUAL  ACTIONS  (PICA) 

As  indicated  by  the  introduction*  the  basic  objective  of  PICA  was  efficient 
problem  identification.  In  order  to  achieve  this  goal,  various  elements,  such 
as  specificity,  relevance,  and  significance,  were  incorporated  to  guarantee 
optimal  results  from  the  proposed  quality  system.  Management  already  knew 
that  contractual  actions  were  not  perfect.  What  they  needed  to  know  was  what 
the  precise  problem  was  and  where  it  was  occurring.  Supervisors  wanted  to 
stop  training  all  buyers  if  90  percent  were  compl^ng  with  policy  and  procedures. 
Therefore,  the  PICA  data  file  was  designed  to  produce  specific  information; 
i.e.,  errors  could  be  identified  by  section  and  by  contracting  officer. 

Another  element,  relevance,  was  essential  to  the  success  of  PICA.  The 
attributes  selected  for  review  had  to  apply  to  a significant  portion  of  the 
contractual  actions.  In  order  to  invest  resources  in  an  efficient  manner, 
managers  needed  to  identify  procurement  tasks  which  would  occur  frequently  in 
the  sample  being  reviewed.  They  did  not  like  to  waste  their  time  on  problems 
that  arose  in  only  very  rare  Instances.  ' 

Finally,  significance  became  an  important  component  of  PICA.  Managers  wished 
to  evaluate  the  iiit>act  o.*  errors  on  the  procurement  process  ar.d  to  determine 
If  corrective  action  would  be  worthwhile.  Among  their  considerations  were  the 
likelihood  of  a protest  or  dispute,  the  question  of  equity,  and  the  identifica- 
tion  of  substantive  errors  as  opposed  to  errors  of  form. 
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SELLING  PICA  - . . " 

Although  PICA  was  designed  to  assist  managers,  the  monitor  at  OC-ALC  felt  the 
success  of  the  program  depended  upon  how  well  it  was  received  by  buyers/ 
contracting  officers—the  "victims".  In  order  to  sell  PICA,  the  monitor  sent 
a representative  to  the  buying  divisions  to  give  informal  briefings.  This 
personal  touch  definitely  contributed  to  their  acceptance  of  the  program.  In 
fact,  the  PICA  analyst  has  become  an  ombudsman  of  sorts.  Buyers  frequently 
request  her  assistance  in  solving  problems  they  are  having  with  local  policies. 
The  results  of  this  dialogue  have  been  amazing.  At  least  ten  different  policy 
areas  were  redefined  in  the  first  eight  months  PICA  was  in  operation.  For 
example,  several  ambiguous  general  provisions  were  presenting  problems  in 
interpretation.  The  language  of  seme  of  the  clauses  conflicted  with  the 
Instructions  spelled  out  in  the  applicability  statement.  In  order  to  solve 
this  problem,  the  ALC  requested  AFLC's  assistance  in  clearing  up  these 
ambiguities.  As  a result  of  the  request,  AFLC  submitted  an  ASPR  case  to 
correct  the  applicability  statement  for  ASPR  clause  7-104.15,  "Examination  of 
Records  by  Comptroller  General." 

Another  Illustration  of  PICA'S  effectiveness  in  clarifying  fuzzy  areas  of 
policy  is  verification  of  bids/offers.  When  the  error  rate  on  the  attribute 
measuring  this  task  first  exceeded  the  acceptable  quality  level,  the  problem 
was  discussed  in  detail  in  the  monthly  PICA  analysis.  Clear-cut  guidance  was 
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resulted  froa  these  sessions  as  shorn  by  the  substantial  inprovenent  in  the 
error  rate  on  that  attribute  from  20  percent  in  July  1976  to  6.3  percent  In 
February  1977. 


given  on  handling  suspected  mistakes,  and  all  buyers  who  incorrectly  handled 
a suspected  mistake  were  asked  to  discuss  the  situation  with  the  ALC  monitor 
for  Public  Law  85-804  and  ASPR  2-406.4.  Some  very  productive  learning 


( 
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As  demonstrated  by  the  above  examples,  PICA  has  become  a learning  device  for 
both  the  buyer  and  the  reviewer.  The  buyer  is  forced  to  take  time  to  study 
ASPR  and  other  procurement  directives,  and  the  reviewer  is  able  to  see  the 
obstacles  certain  policies  are  creating. 


' i 
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When  developing  the  inspection  sheets,  OC-ALC's  procurement  managers  elected 
to  use  attributes  which  had  received  criticism  from  various  inspection  teams. 
All  available  inspection  reports  were  reviewed,  and  those  criticisms  which  were 


considered  serious  enough  to  merit  management  surveillance  were  incorporated  in  p 
the  attribute  checklists.  Also,  quiestions  were  developed  to  measure  compliance  .• 

with  newly  implemented  policies.  The  attributes  are  objective  questions  which  ( 

can  be  answered  "yes"  or  "no".  For  exaigile,  "Are  proposed  procurements  ^ 

synopsized  not  later  than  the  date  of  issuance  of  the  solicitation?"  Although  • 

not  all  questions  are  applicable  to  all  actions  reviewed,  care  was  taken  to  ; * 

insure  that  the  attributes  were  of  a general  nature  and  would  apply  to  a large  v 

enough  sample  of  actions  to  be  statistically  valid.  To  better  achieve  this 
objective,  separate  checklists  were  developed  for  each  type  of  contractual  > 

action;  i.e.,  small  purchases,  contracts  over  $10,000,  and  modifications.  The  i 

checklists  illustrated  in  Exhibits  1 though  4 remain  in  use  for  a 12-month  S 

period.  At  the  end  of  that  time  they  are  reviewed  and  purged  of  invalid 
attributes.  Cuestions  which  are  deleted  include  those  which  (1)  have  not 
exceeded  the  acceptable  quality  level,  i.e.,  are  not  consider^  to  be  ‘ ( 

discreMnt,  (2)  do  not  have  a large  enough  sample  for  statistical  validity,  N 

and  (3)  are  no  longer  useful  measures  of  quality  regardless  of  the  error  rate  , i 

or  sample  size.  As  a result  of  the  first  year's  review,  63  attributes  were 
deleted,  35  were  added,  and  18  were  retained.  This  major  revamping  of  the 
checklists  was  due  to  the  fact  that  PlCA's  first  year  of  operation  was  largely 
experimental  with  the  majority  of  the  attributes  proving  to  be  unsuitable. 

STATISTICAL  ANALYSIS 

In  the  PICA  system  randomly  selected  contracts  are  ertamined  to  determine  if 

specific  procurement  tasks  have  been  performed  correctly.  These  tasks,  which  ! 

are  called  attributes,  are  rated  as  either  discrepant  or  not  discrepant  by 

the  reviewer.  The  purpose  of  the  exercise  is  to  compute  the  error  rate  of  the 

sample  and  to  estimate  the  error  rate  of  the  total  population.  This  information 

is  printed  out  in  the  format  shown  in  Exhibit  5.  Statistical  analysis  of  the 

sampled  attributes  is  based  on  the  binomial  distribution  which  is  a discreet 

theoretical  distribution  of  a series  of  independent  yes/no  responses.  The 

computer  program  uses  sample  data  on  the  number  of  discrepancies  for  each  ( 

attribute  and  the  number  of  times  the  attribute  is  sampled.  A likely  upper 

limit  (90  percent  confidence)  of  the  error  rate  for  the  total  population  is 

computed  for  each  attribute  based  on  the  sample  data.  As  the  sample  size  ' 

increases,  the  probable  error  rate  for  the  entire  population  of  the  attribute  ^ ' 

approaches  the  true  population  error  rate.  For  samples  of  30  or  larger  a very 
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monthly  pica  review 

program  was  developed  to  determine  the  number  of  files 
0 be  pull^  tr*’*  total  population  of  files  In  order  to  Insure  a valid 

1ndllidiial*ttfi^^^^  "°?i***  review  covered  2081  actions  and  19,118 

^^4  appropriate  attribute  checklist  is  completed  for 
iJ  *'*  ‘‘^scussed  with  the  contractlng^flcer/buyer. 

convinced  the  reviewer  that  the  buyer  has  a^ 
valid  position  and  the  discrepancy  Is  removed.  In  fact,  the  dialogue  between 
£ the  ^^r/contractlng  officer  has  definitely  contJibutedIo 

the  hiaanizatlon  of  PICA  by  promoting  friendlier  attitudes  among  the  "revlewees", 

review  are  furnished  to  all  levels  of  supervision  on  a 
"onthly  analysis,  as  represented  In  Exhibit  6,  provides  a 
overall  e^r  rate  as  well  as  a section  level  error  rate  for  each 
^"tludes  a discussion  of  the  problems  encountered 
^ ^<r  suggested  sol^lons.  This  monthly  analysis  Is  possible  because  PICA 
ISrAT^?j ^ Jesuits  of  the  monthly  review  are  Input  Into  the 

2SlLS^£<r*lI^/  report  Is  produced  which  states  the  90^rcent 

^fl^ce  and  sample  error  rates  for  each  attribute.  Both  overall 
directorate  and  todividual  section  error  rates  are  computed.  An  arbitrary  rate 
Is  selected  as  ^ acceptable  quality  level  (AQL)  and  directorate  perforronce 

Co'^otlve  action  is  directed  for  those 
**1ch  exceed  the  AQL.  This  action  ranges  from  simply  discussing 
tod  SUSdlris.^  analysis  to  actually  changing  unw^kable  pollctes 

developed  to  assist  buyers  who  are  having 
^ficul ties  with  specific  procurement  tasks.  These  courses  are  part  of  the  ^ 

PICA  ^ The^DWf^"V'^ri^"^J[^^^  Program  which  operates  In  tandem  with 
PICA.  Tw  Directorate  Training  Monitor  Is  advised  of  significant  tralninq 

^d<  ‘***^9ns  courses  to  cover  thesJ 

needs.  Brief  video  tape  cassettes  are  used  extensively.  Instead  of  having  an 

on  the  same  subject,  the  presentation  Is  tapto 
tod  replay  In  the  various  organizations  which  PICA  has  Identified  as 
having  a need  for  refresher  training  on  a specific  subject. 

PICA  RECOGNITION  PROGRAM 

To  add  incentive  to  Pitt  the  Director  has  established  a recognition  program 

quarter  the  section  with  the  lowest 
5’''®"  *"  program  Is 

dMignated  the  PICA  ^e  Award  and  the  Snoopy  Flying  Ace  figure  Is  used  as 

^ ^I**  *"  objective  selection  criteria  enhances 

^ effectiveness  of  tte  award.  The  theme  helps  make  PICA  more  palatable  to 
buyers/contractipg  officers  while  the  objective  basis  for  recognition  (PICA 

rospectabllity  to  the  award.  The  PICA  Ace  Program  encourages 
co^etltion  among  the  sections  which  In  turn  Improves  the  directorate  error 
rate. 
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ASSESSMENT  - • ^ 5/  • ; 

While  most  of  the  edvdote^cs  of  PICA  heve  been  discussed  prev1otis1y«  they 
need  to  be  reiterated.  The  purpose  in  Instituting  PICA  was  to  help  procurement 
nugers  do  a better  job  of  managing.  The  monthly  analysis  is  a management 
Information  system  in  the  truest  sense  because  it  enables  a supervisor  to  ‘n 
determine  pHorities.  At  a glance  he  can  isolate  the  procuroKnt  tasks  which' 
are  not  being  performed  correctly  in  his  section.  If  the  error  rate  exceeds 
w acceptable  quality  level,  he  has  various  alternatives  available  to  improve 
the  quality  of  his  buyers' contractual  dociaaents.  He  can  require  more  extensive 
wiew  of  contracts  prior  to  award,  counsel  Individual  buyers,  discuss 
deficiencies  in  section  meetings,  or  schedule  Proficiency  Enhancement  Training. 

In  addition,  since  PICA  identifies  the  problems  offertng  the  greatest  potential 
for  cure,  the  supervisor  doesn't  need  to  make  value  judgnmnts  each  time  he 
discovers  a *«cient  area;  if  the  task  is  covered  by  PICA,  he  gives  it  maximta 
attention.  Thus  PICA  saves  training  hours  and  supervisory  effort  by  focusing 
2**  problems  Insteed  of  tek1n9  e shotQun  ipproach.  It  also  serves  as  a 
training  aid  on  major  policy  changes.  Since  some  PICA  attributes  are  designed 
to  measure  compliance  with  newly  implemented  policy,  the  monthly  review  is  a 
constant  reminder  to  buyers  and  helps  speed  up  the  learning  curve. 

Furthermore,  PICA  can  be  used  in  conjuiction  with  other  management  indicators 
which  measure  contractual  actions  completed  delinquent,  processing  time  of  buy 
actlOTS,  overage  purchase  requests,  and  efficiency.  The  supervisor  can  compare 
his  ^yers  performance  in  two  Important  areas—timeliness  and  quality.  For 
the  firtt  time  he  is  able  to  determine  if  he  is  sacrificing  quality  for  the 
sake  of  timeliness  or  vice  versa.  Of  course,  both  are  required  for  a successful 
procurement  operation  but  trade*offs  must  occasionally  be  made  between  the  two. 

The  most  recent  advantage  of  PICA  to  come  to  light  is  its  capacity  for  reducing 
Ins^ctlon  and  reporting  burdens.  The  workload  inherent  in  preparing  for  a 
visit  by  the  AFLC  Inspector  General  was  considerably  reduced  because  of  PICA. 
Instead  of  performing  Independent  reviews  of  an  unscientific  sammle  of 
contractual  actions,  the  self-inspection  group  was  able  to  consult  PICA 
analyses  to  determine  how  we  were  doing  In  areas  susceptible  to  review  by  the 
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14.  Art  rtvttU  ftr  mtt  mi  ftotl  tfftrt  ttto  ttly  mtt  tcptfUtltts  hmm  tctttll/  met  mi4 
tm  tfttr  tll  trttt  of  dluprintt  ttvt  bttt  molvmr  Art  til  offerort  withto  tm  ctmotitfvt 
rttft  f Ivw  tt  tppMtotlt/  U proptBt  tf  tmrt  trt  ctotfm  ctmttltttT  (AFU/lff  ltr.  2S  Ito* 

15.  mtt  S rittl  PrtSIttt  «m  mfSeltats) 

If.  It  tm  rttttt  ftr  tm  tttrrvpttttottrtt  of  SWf  toci  twfm  tt  cm  Of  Hm  1S01.  *Amtrtct 
•f  itoi*T  (IfOl  lO^lf  Mi  SIMU  IS  f taditf  OR.  IS  Apr  • 10  toy  IS) 

IS.  m prlct  Wfotlottot  mmmrmimm  prtperm  ir  N/tof  iirtoton  ttoqtitol/  ttpport 
totoBvIttttot  tf  ftlr  tm  rtMttaSlo  pHctT  (PfOl  70-11;  PP  Ltr.  17  Otc  75.  Prlcltf 
ifltilttltt  tm  Smportitf  Dnr— tottm;  tm  OC-AU  IS  Ftoiltf  OS.  f-24  Hdv  7S) 


II.  It  tMm  to  totr  tmt  lit  tffcftr  tmt  it  tccomatct  mtii  ppoi  ioast 

If.  Art  irfMt  Ptrcmtt  rtmttU  prKtssei  cmmitltt«1/T  (OC-AU  IS  Fimitf  M.  f-24  fot  IS) 


If.  Art  ctmtt  pracettrtt  ftllotm  rttotivt  tt  prtMttrS  toilet  tf  tpptnttl/  stccttsftl  tfftrtr 
tt  IMlI  tostattl  Mt-Oflitsl  (asm  S-50.2(S)] 

10.  to  rospMM  ttto  tt  SJA  ctmu  «tot  »ppnmrUU7  (PPOI  lOAl;  OC-ALC  IS  Fimitf  002. 
PrwtrtM'fmtM)^*'*  ^ ^ tortlclptttot  tf  im  suff  toift  AtotcaU  to  tm 


a.  !•  Cttritoitlot  ettpittm  wItA  tm  Ittt  Httfrr  timm  tm  i 
tm  OC-AU  IS  Fimitf  002.  f-24  mv  7s) 

a.  ftn  tm  mmri  rtpilrt  ntoltcltt  tf  t dItetoMrt  tUUtti 
If  ftf  IS.  Cta  AcctMtIfif  StBmirdi,  fttclottrt  StBt—U) 


m tote  It  tot  mtr  (PPOI  I0-S5 
ptr  Asm  }-l20St  (PPLtr 
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EXHIBIT  4 


Vtttl  SlCfT 

ucio  is; 


u 


T ' • . _i  ^fjsn 


refyi 


m.  I»  Ut  r«9[ilr«4  ^tHkiry  sUt>ne«it  •s\irt4  «•  cm^»Citftc  VBlIclUtfouf  for  fils  olitdb  Uf 
■pfirw  t ly  ocvd  4oir^F  ?0-M  om*  OC-Atf  14  fimen^  k4»  9-7<  Mov  7$) 


W.  I«ti»  re«ifrtd  tfellwtry  tutavH  lo  ««Ucluit9n  far  or^t  lata  tayrct  ti^r 

(ffOl  l»-SS  OC-AiC  14  rt<idt»«  C4.  9-7«  Kov  74) 


M9.  lata  Urn  rc^trod  drllvovjr  wkcdoU  ia  tat  lallcfuttoa  coa'atF  U tae  fV  otH  doUf 
(MMU  14  nadlof  002.  19  Sea  - « OlC  H$ 


IM.  drv  proetdiaat  caaufatd  0t>AtC«Urt9  70>I9  rolloHcd  If  ft  it  atried  taltaklt  far  ti» 


McftJftaf  ttnrlcrt  caatroctl  (ft  IttUrt,  S A 22  Apr  7o.  fnearpaat  af  fretarvaitaa  asa 
Mtaf  Strvicttl  ^ 

1M.  Is  tfatly  fallaaHip  feltiotod  oo  »n1tciutf*»r  (fff!  fO>4S;  ft  Itr,  If  fiec  7S» 

fallat^aa  Actiou  at  Overdat  OHOUtioMi  tad  QC'AlC  1C  fit^iog  0f»  9-29  aev  79) 


m.*  Art  aids  and  offtrt  vtrlftod  iditrr  tae  file  fntflcatct  wch  actlaa  sfowld  At  Ukaaf 
(ff  llr.  9 Mir  74*  Coatractar  taotiics  far  lletlaf  MOtr  hiaik  Urn  a«id  Mlttafcat  la  tfd) 

ttr.  to  prict  atdaCfailoa  a>aD»aiA<«n  prfcjrad  by  Ua  buyfat  dfvHloa  adrqMaCtly  support 
ddttmiaatloa  of  fair  aad  renoMblt  prKt?  (Ffr.;  74-11;  ff  lir.  if  Ok  75,  f<‘lclfi9  fitspc#- 
»fbllft<as  aad  Sopportl^Q  OociaMatioa;  aaa  X->C  IZ  Fiadla;  05,  §.29  tov  75) 


Mi.  Is  tbt  raasaa  for  tba  ^alklUtfai  of  aaotts  frtBda.y  aat  savea  sUUd  la  tbi  9wa 
(lA-AU  K fiadiat  001,  19  Sap  • 9 Oct  74) 


Mi.  Is  OHord  to  atbtr  Uia  lo«  af farar  aHap  la  OKsiteca  atr*  ffOt  70-557 


IM.  Art  arptat  pKVbisa  re^^asu  proccssad  ci^ftlsaslyT  {OC-ALC  16  fiadlao  01, 


to.  75) 


111.  Is  ceordlaatloa  csapitttd  wftb  tbt  it«a  Jdaiptr  t«tica  fit  at«d  diU  Is  aot  ^17  (ffOl 
7t-SS  aad  0C>AI£  16  ftadlnp  007,  9-24  Ho.  75) 


112.  Is  flit  eroptrly  dncititoltd  i*tr»  rrcit  It  <lii*fi«ied  froo  4 fa  5 aad  art  aettstary 
caadftlaas  prttcacf  (ff  Itr,  15  Oct  74.  ««4I  5yitt«  Cjrcit  5 AcCioas  $10,000.) 


111.  to  tbe  laspcctfoa  criteria  tacladed  la  tbt  coatract  coaforw  arltb  tbt  rcqulraarats  af 
tbi  ft?  If  aot,  bit  QC/MI  ipprancd  mj  ebaapM?  (ASfi  14-201) 


119.  Oats  ceaCract  flit  coaUia  dacaaeautloa  as  rtqvlrtf  by  ASPS  Saoplaatat  Mo  2,  paraprapb 
62*102.1  aad  fPOJ  12*17  Art  flits  la  correct  sta^ract?  Art  tbera  loosa  dociaants.  aoast/.'dar 
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o 
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U. 


CONTRACT  STANOBRO  INSPECTION 


V • ' 


rr-r* 

• 1 ' SYNOPSIS  ACCURATELY  REFLECT  THE  PONT  t OPTION'» 


u 

o 

o 

C' 

L' 


PP/SECT 


' MONTHLY  DATA 

SANP.E/R  SOr:CONF.LIH  NO.APP' 


' CUOULATIUE  data 
SANP.E/R  AOIJCOHF.LIN  NO.APP 


I 


L 


PP  TOTRL 

.000 

.068 

HH 

.024 

.048 

165 

Zflfi 

.000 

.867 

3 

" ^*.000 

.231 

13 

ZflB 

< .000 
.000 

.273 

11  - 

.074 

• ^ ^ w 

27 

ZER 

1.000 

1 

' .05?  . 

.235 

17 

ZEB 

.000 

.300 

to 

.000 

.086 

35 

ZIR 

.000 

.667 

3 

r.ooo 

.300 

10 

ZIB 

.000 

.HOC 

5 

. .000 

11 

IfHB 

,.oco 

.500 

4 

, .000 

. 176 

17 

Hfilt 

.000 

1.000 

1 

.143 

.571 

•y 

UCR 

.000 

.333 

.6 

.000  * 

. 1 15 

26 

UCD 

.000 

. .000 

0 

.000 

1.000 

-> 

PPOPnSEO  PPCIIT 

svNOPsiz: 

ED  JILT  ! 

?SUR«CE  OF 

SOLICITATICH"* 

tIOMTHLV  DRTR 

CUmiLRTI  UE  DRTR 

PP/i^ECT 

5;rmp.e/p  <>o:;cohf.li« 

HR.RPP 

SR«P.E/P 

9o::coriK.LiM  ho. 

RPP 

PP  TOTRL 

.OHO 

.120 

50 

.013 

.033 

15? 

ZRR 

.000 

.667 

3 

.000 

.250 

« o 

ZRB 

. 100 

.'♦00 

to 

.045 

. 1C2 

.?2 

ZEfi 

.000 

1.000 

1 

.UOO 

.214 

14 

ZEB 

.000 

.300 

10 

.000 

. 1C7 

28 

ZIfi 

.000 

.667 

3 

.000 

. 300 

10 

ZIB 

.000 

.'♦00 

5 

.000 

. 333 

? 

i:rb 

.000 

.500 

4 

.000 

.231 

13 

!iRri 

.000 

1.000 

1 

.000 

.333 

6 

NCR 

.167 

.667 

6 

.042 

• 1 67 

24 

UCB 

.000 

.000 

0 

.000 

1 .UOO 

1 

KC 

.000 

.000 

0 

.000 

.333 

6 

KS 

.000 

.286 

7 

.000 

.214 

14 

3 

ASPP  EXCEPTION 

1-1003. 1<IV>  TO 

’V/H0PSIZI4«i 

PPOPOSED  PPCriT 

PPPPOPPTRTELV  REPLIED? 

MOMTHLV  DRTR 

CUmiLhTILip  DRTR 

PP/SECT 

srmp.e/p 

.CONF.LIh 

HO.RPP 

SRMP.E/P 

?o?.cohf.l:p.  mo. 

HPP 

\'A 


/# 
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• EXHIBIT  6 

DEPARTMENV  OF  THE  FORCE 

0«lA««CMft  «.!•«  A«« 

»tN«C*  Mtm  9 0tn.9  m»*t  »»i«s 


' DOT 
•r<  rrL 


# < A-* 

• V • * : 
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.rf  • . . • % vi‘  » f-t  ^ 


Prograii  for  Improvement  of  Contractual  Actions  (PICA)  Section  Analysis 
as  of  May  1976 


PPZ 


PPO 


PPK 


o 


1.  PICA  data  for  May- 1976  (twth  nonthly  and  ciAwlative  totals)  Is 
V fonrded  for  your  review.  Please  refer  to  PPC  letter.  16  Mar  76,  Program 

Improvement  of  Contract  Actions  (PICA)  Section  Analysis  for  Oct.  Mov.^  . 
. Dec  1975,  for  instructions  on  interpreting  the  data.  ^ f 


2.  Management  Review  items  for  May  will  be  attributes  9,  11/109  (combfnedl 
20/76/114  (combined).  24/116  (combined).  29.  30/81  (conbined).  and  49.  . - ( ; 

■*  The  attributes  which  have  become  statistically  valid  since  our  last  report  - 
are  30,  68.'  109,  and  116.  The  previously  reported  attributes  idiich  are  no 


lornger  discrepant  are  20,  54 

, and  110.  (Good  work!!)  As  you  can  see  By 

ti» 

following  statistics,  we 

are  li^^rovlfig  In  all 

but  four  areas: 

U- 

brar 

Error 

Error 

Error 

Error 

Error 

Rate 

Rate 

Rate 

Rate 

Rate 

Rate 

y 

As  of 

As  of 

As  of 

As  of 

As  of 

As  of 

U- 

Attribute 

7512 

7601 

7602 

7603 

7604 

7605 

7 

SO" 

WT 

WT~ 

ATT 

TTT 

3nr 

9 

-0- 

-0- 

6.3 

4.8 

31.0 

33.3 

. \ 

14 

20.5 

20.4 

17.2 

15.4 

11.8 

10.3 

w 

■ ' 

. 20 

13.2 

12.2 

11.7 

11.5 

10.3 

9.6 

• 

29 

3.2 

4.5 

5.3 

12.8 

13.3 

16.2 

30 

50.0 

57.1 

57.1 

57.1 

44.4 

57.1 

49 

33.3 

42.9 

33.3 

28.6 

20.7 

17.1 

L/ 

54 

45.5 

2B.6 

20.0 

16.2 

12.2 

8.7 

68 

-0- 

-0- 

5.6 

14.8 

12.5 

10.3 

76 

12.5 

25.0 

28.0 

23.1 

21.8 

21.5 

( ) 

78 

-0- 

9.5 

11.8 

12.2 

11.7 

11.3 

109 

7.7 

5.9 

14.3 

14.8 

16.1 

21.2 

’ 

no 

7.5 

.7.5 

11.0 

11.2 

10.3 

9.0 

114 

‘14.7- 

•13.1 

13.5 

11.9 

11.4 

10.5 

\J) 

116 

-0- 

4.8 

12.5 

14.8 

15.6 

13.9 

3. 

The  error 

rate  for  attribute  9. 

"Handling  of 

First  Article  lAM 

— 

Current  Directives," 

continues  to  rise. 

Problems 

1 found  in  May  are  the 

1 ■ > 

e ones  found  In  April.  Since  the  attribute  was  discussed  In  our 

9 Jun  76  letter,  we  expect  Improvement  in  June  and  July  files. 


4. '.The  error  rate  for  attribute  29,  “File  Properly  Oocunented  Where  CycU 
Change  from  4 to  5 and  Necessary  Conditions  Present,"  did  not  improve  in 
May  as  predicted.  It  is  apparent  from  the  May  data  that  four  sections 
have  successfully  implemented  ?P  letter,  6 Oct  75,  J041  System  Cycle  5 j 

Actions  Under  $10,000,  i.e.,  they  have  no  errors.  Pe^ps  the  .renaining 
si*  sections  could  improve  their  performance  by  adopting  the  procedures 
followed  in  PPZAA,  PPZIB.  PPHCA,  or  PPWCB. 

JIfjCC  • jCi/f/inf  a/  lif  'Afifipue  Otam  * ( ^ 
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5.  Attribute  30’;"  "Does  Award  Riwjuire  Disclosure  Statement  Per  ASPR 
3-1203?, ■ becaiM*  statistically  valid  this  month.  ASPR  3-1203  requires 
that  the  contractor,  with  some  exceptions,  submit  one  of  the  certifica- 
tions outlined  in  ASPR  7-2003.67;a)  with  each  offer  tAich  could  result 
In  a negotiated  contract  exceeding  SIOO.OOO.  In  addition,  the  PCO  Mst 
obtain  a «tateinent  from  the  AGO  regarding  the  adequacy  of  the  contractor's 
disclosure  statement.  In  accordance  with  PP  letter,  19  Feb  76,  Cost 
Accounting  Standards , (CAS)  Disclosure  Statements.  PPF  is  asking  for  the 
ACO  statement  in  each  request  for  DCAA  audit  and  field  pricing  assistance 
and  is  insuring  that  the  ACO's  position  is  discussed  in  the  Price 

t Negotiation  Memorandum.  However,  it  is  the  PCO's  responsibility  to 
' obtain  the  ASPR  7-2003.67(a)  certification  from  the  contractor.  The 
latter  requirement  is  presenting  difficulties  to  buyers/PCOs.  !f 
solicitations  are  made  by  SF  33.  the  Disclosure  Statement  - Cose  Accounting 
Practices  and  Certification  (1975  Dec)  (OCAICF/B-5  and  B-6)  is  included. 
Ilien  the  buyer  solicits  by  TUX  in  contemplation  of  attard  by  BOA  order. ^ 
he  or  she  can  easily  forget  to  request  the  disclosure  statement  certifica- 
tion.  In  order  to  correct  this  situation  PPC  has  drafted  a disclosure 
statement  certification  which  is  to  be  added  to  local  BOAs  by  supplemental 
agreement.  In  addition,  we  are  advising  AFLC  and  the  other  ALCs  of  the 
• need  for  this  addition  to  the  Uniform  BOA  Format.  In  the  meantime,  buyers 
will  need  to  include  a request  for  the  disclosure  sUtement  certification 
in  solicitations  expected  to  result  in  a negotiated  BOA  order  over 
$100,000.  OC/SPF-10-1  and  OC/SPF-10-2  could  be  included  in  written 
solicitations  or*mailed  at  the  same  time  a TUX  solicitation  is  transmitted. 

6.  Attribute  68,  "Small  Business  Sheet  in  Error  or  Incomplete.”  has 
exceeded  the  minimum  acceptable  quality  level  (AQL)  for  the  pasfthree 
months.  Mistakes,  which  have  been  very  minor,  include  incorrect  PR 
estimate  in  block  1,  no  PR/MIPR  number  in  block  2,  erroneous  information 
in  block  4.  and  inappropriate  negotiation  authority  in  block  8. 

7.  Problems  have  also  surfaced  with  attribute  109,  “Determine  if  Bids 
and  Offers  Are  Being  Verified  Uherc  Indicated.”  Although  PPC  has  been 
providing  guidance  to  buyers  whose  files  have  discrepancies,  we  feel  that 
additional  emphasis  is  needed.  In  the  past  three  ywrs  at  least  three 
letters  have  been  written  to  provide  guidance  on  verification  of  bids 
and  offers  (PP  Ur,  7 fib  73,  Verification  of  Bids  and  Offers;  PPC  Ur, 

14  Aug  74,  Verification  of  Prices  on  Suspected  MisUkes;  and  PP  Ur. 

9 Mar  76,  Contractor  Requests  for  Relief  under  Public  Law  85-804  and 
Mistakes  in  Bid).  The  first  two  letters  were  rescinded  because  they  .3^4 
served  their  purpose.  Unfortunately,  it  appears  that  tt»  adage,  "out 
of  sight,  out  of  mind,"  applies.  Therefore,  the  following  conments  are 
offered  to  assist  buyers/PCOs  in  discharging  their  responsibilities. 

ASPR  2-406  requires  that  when  a buyer/PCO  suspects  a mistake  in  a 
contractor's  bid,  he  shall  request  the  prospective  contractor  to  verify 
his  price,  "calling  attention  iio  the  suspected  mistake."  The  adequacy 
of  this  verification  is  addressed  in  Comptroller  General  Decision  B-177405 
which  states,  in  part:  ' 


. »ii 


\ 


398 


- \-  -•j.'sTx 


•.  N 


V. 
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* ’'equest  for  verffkation  to  conforn-  to  tfv*  good 
‘ «\Pecied  of  the  Governnienf?  contracting  ‘ ^ 

Son?Ml  ™ **  infonnative  as 

^sible  vonceraiiig  the  pertinent  factors  indicating  to  the 
contracting  officer. that  an  error  was  made  in  the  bid." 

^ wen-doc»«ented  as  to  the  efforts  Uken  by  the 
U^ba^s  for  ^“sP«cted  mistake,  including 

sisjL.  «PR.p.rt'^.  ^ c«5rtr“ 

coSSuM  S'f^  " '»«'"*-  fth 

* * . • '.-  -r  ♦•  * 1 * < 

8.  ^^Mlly,  attribute  116,  "Policy  of  PPOl  70-79  and  PP  Letters  FollokMi 
in  Procurement  of. Overhaul  Parts  Repair  Kits  " has  hero«. 

buys  is  rcBcmbcrl  119  to  Include  the  reouii'ed  in 

(OC/SPF/PO-2-3)  an^JesuTtant  awrtl  (0C/SPF/P0- 
PCo]*shoild*'tJ®I?rr^"l^"?  procedures  have  recently  been  chanoed,  buyer^ 

discrepant  attributes  have  been  discussed  previously  With 
collected,  55  of  81  PICA  attributes  tale^ilie  ” 
statistically  valid.  Of  the  55,  only  12  exceed  the  acceptable  AQL  of  101. 
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fAIL-STD-1567(USAni  THE  BILLION  ooaMtMOff 

w;  i.-r  ’ i-  . > v'f  - ^ -Hy.crq  .( 

if-'  i' a ‘i.il  isi;<^ *  *7^ 

Don  A.  Hoore  . 

Kirtland  AFB,  ftew  Mexico 

INTRODUCTION 

This  poper  describes  the  cycles  of  needs,  research,  and  implementation 
efforts  neccssory  for  the  development  of  Ml L- STD-1 567(USAF)#  Work 
Measurement,  and  the  use  of  ^L-STD-1567(USAF)  to  achieve  cost 
reductions  in  the  acquisition  of  major  USAF  v/e<^n  systems.  This 
description  is  presented  for  two  reasons*  The  Hist  is  to  demonstrate 
the  potential  of  MIL-STD-I567CUSAF)  for  immediate  improvement  in 
manufacturing  perfotmonce.  The  second  is  to  provide  a prototype 
success  story  of  a procurement  research  project.  The  moral  of  the 
prototype  procurement  research  success  story  is  that  for  procurement 
reseorch  to  be  successful  requires  o commitment  and  dedication  to 
continuing  reseorch  ond  implementation  efforts.  This  effort  olso  requires 
the  education  of  manogement  sufficient  to  create  the  climote  that  mokes 
possible  implementing  the  reseorch. 

• ^ ^ It,. 

The  model  used  to  describe  the  research  process  and  used  os  the  formot 
to  describe  the  efforts  to  develop  and  implement  MIL-STD-1567(USAF) 
is  depicted  in  Figure  1 below.  The  events  described  in  the  format 
so  depicted  are  not  necessarily  in  true  chronological  o*der.  As  in 
applying  all  models,  there  is  in  reolity  o blending  of  the  steps  that 
ore  depicted  as  discrete.  Needs,  implementation  efforts,  oikJ 
research  are,  in  some  coses,  integrated  arxl  indistinguishable.  This 
reinforces  the  moral  of  our  story  — that  procurement  reseorch,  to  be 
successful,  requires  continuing  research  and  implementation  efforts. 


Rgurc  1 


This  model  depicts  on  iterative  cycle.  Whether  or  r»t  the  iterative 
process  converges  depends  on  the  extent  to  which  each  step  of  the 
process  continues  to  focus  on  the  end  results  that  must  be  achieved 
to  truly  meet  the  originol  need.  Therefore,  the  model  for  a 
successful  research  process  is  the  one  depicted  below  in  Figure  2. 


The  results,  in  the  cose  of  MIL*STD-1567(USAF},  would  be  the 

the  costs  of  oct^iring  major  Air  Force  weapon  systems 
through  improving  contractor  direct  monufachirirtg  labor  productivity 
ortd  other  improvements  incidental  to  the  disciplined  use  of  an 
effective  work  measurement  and  methods  improvement  system. 


Simply  stated,  the  need  is  to  lower  weapon  system  hardware  costs.  The 
Air  Force  and  Deportment  of  Defense  concern  for  the  spiraling  cost 
itscreoses  fosters  almost  continuous  study  and  research  in  how  to  reduce 
or  better  control  such  cost  irtcreoses.  The  need  has  been 
**ocerboted  by  relative  ond  absolute  declines  in  the  proportion  of 
the  budget  and  national  product  devoted  to  Defense.  Continuing 
inflation  plus  the  steady  increase  in  personnel  costs  during  the  last 
decade  further  increased  the  pressure  to  get  the  most  out  of  our 
hardware  dollars.  Furthennore,  it  appears  that  the  public  perception 
of  the  efficacy  of  Defense  spending  resulted  in  decreased  support  for 
such  spending  over  much  of  the  lost  decade. 

In  reality, the  need  oppeors  to  be  somewhat  greater  than  simply 
lowering  hardwo  e costs.  The  need  appears  to  be  to  demonstrate  the 
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efficacy  of  Defense  spending.  In  fhe  area  of  new  Air  Force  weapon 
systems  acquisitions,  this  meons  r>ot  only  must  costs  be  reduced  or 
controlled,  but  the  management  must  be  demonstrobly  effective  and 
efficient. 

Reseorch  - > r *.  -..r.-.'’  : •••  -tv  ’(i' ‘j;*!  o.*- 

To  dbbreviate  somewhat  a rother  lengthy  story,  only  a sampling  of 
key  studies  will  be  preserved.  • r L r' 


In  1970,  the  Monufbcluring  Committee  of  the  Aerospoce  Industries 
Ass'*ciation  conducted  a sur/ey  from  which  the  following  .conclusions  . - 
were  mode:  , ; 'n?-.  . .if  r’ii-f 

i rt 

"le  Among  respondents  there  is  almost  universal  use  of  stondards 
for  lobor  plannir^g  emd  control  of  production  lobor^  and  in  addition 
two-thirds  of  the  respondents  use  standards  for  meosuring  some  portion 
of  the  indirect-type  or  manufacturing  support  activitiese  Those  who 
use  stondarxis  for  other  than  f>roduction  Icbor  vary  widely  in  the  types 
of  activittes  covered  by  standards*  There  appears  to  be  considerably 
less  oc;ivity  anchor  success  in  meosuring  indirect  Icbor. 


"2.  Organizational  placement  appears  to  have  little  to  do  with 
respect  to  stondords  effectiveness  05  shown  by  reported  savings  vs 
costs  ratios.  Among  respondents  who  claim  knowledge  of  their  savings 
vs  costS/  ratios  vary  from  2:1  to  over  5:1. 

•3.  Companies  with  highest  savings  vs  costs  ratios  tended  to 
measure  by  group  ofKl  also  tended  to  use  standard  time  data. 


■4.  Ldbor  performance  reports  on  a weekly  basis  are  by  far  most 
common. 


"5.  Allowances  for  personal,  fatigue  arwi  delay  time  average 
around  13%  with  none  reported  below  5%  and  none  above  22.5%. 

•6.  It  is  envisioned  that  in  the  future  standards  will  be  used  more 
extensively  for  such  purposes  as  computerized  shop  loading,  product 
and  equipment  design  evaluation,  orKJ  irxlirect  lobor  measurement. 

■7.  SlorKfards  are  normally  communicated  to  supervision  and  produc- 
tion workers.  One-third  of  the  respondents  use  operator  performance 
to  standards  for  disciplinory  action. 
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“8.  Seventy-five  percent  of  the  respondents  use  electronic  dota 
processing  terminols  for  doto  collection  and  o like  percentage  provides 
for  non-productive  labor  and  delay  reporting. 

"9.  Among  the  respondents,  obout  72%  of  production  Idbor  hours 
ore  covered  by  engineered  standards  with  a marked  indication  of  - — J 
planned  increase  of  coveixige  and  no  planned  decrease. 

•lO.  Data  from  respondents  indicated  that  personnel  responsible  for 
estoblishing  and  maintaining  standards  systems  tend  to  be  somewhat 
satisfied  with  current  techniques  end  collectively  envision  a need  for 
more  of  the  same.  This  may  be  en  unrealistic  apprxsach,  particularly 
with  the  low  production  c^ontities  associated  with  most  current  aerospace 
programs.*  0)  ^ 

■ - • ^ li*.  -I  - 

The  Lyon's  (for  then  Brigadier  General  Herbert  A.  Lyon,  Deputy  Chief 
Stoff,  Systems,  Air  Force  Systems  Command,  Study  Director)  Study, 
formally  known  as  the  Air  Force  Production  Manogement  Study,  was 
also  conducted  in  1970.  One  of  that  study's  conclusions  ond  recom- 
mendations was,  "Controctuoi  mstraments  hove  not  contained  definitized 
requirements  for  effective  control  of  the  production  process."  (2) 

As  the  result  of  the  Lyon's  Study,  MIL-STD-1528(USAF),  Production 
Management,  was  created.  One  unelaborated  requirement  of  this 
standard  was  for  the  "Maintenance  of  a work  measurement  program."  (3) 

In  addition  to  incorporothg  MIL-STD-1528(USAF),  the  A-10  contract 
contained  a somewhat  more  elaborate  requirement  for  work  measurement: 

“The  contractor  shall  have  and  use  a system  of  measuring  the  effi- 
ciency of  departments  engaged  In  the  manufacturing  process.  He  will 
insure  that  the  system  provides  this  measurement  ot  the  lowest  available 
work  center  permitting  compilation  of  efficiency  rating  to  the  department 
level.  Criteria  to  be  us^  shall  Include:  labor  productivity,  omount  of 
scrap,  amount  of  re-work,  housekeeping  record,  safety  record,  amount 
of  woste,  amount  of  machine  down  time,  shop  loading  record,  planning 
error  record,  and  scheduled  job  completion  record*  Contractor  shall 
also  insure  that  accurate  labor  time  standards  exist  and  are  used  to 
assess  productivity  of  all  departments  engoged  in  the  monufacturing  process. 
He  will  ensure  that  labor  time  standards  are  provided  for  each  operation 
element  of  work  required  of  a worker.  Variance  reports  will  be  issued 
monthly  showing  the  cctual  versus  standard  performance  achieved 
(summation  level)  at  the  lowest  available  work  center.  Compilation  of 
individual  work  center  variance  will  be  used  to  gain  Job  and  t 
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departmental  variance  data*  The  labor  time  standards  along  with 
reasonable  variance  targets  will  also  be  aval  lob  le  to  contractor  cost 
estimating  personrtel  ond  will  form  a bosis  for  menu  factoring  cost 
estimates*  The  controctor  will  ensure  that  this  lobor  productivity  data 
is  provided  as  an  input  to  the  deportmental  efficiency  system* 

Acceptdblo  variances^  in  accordance  with  industry  starxlards,  will  be 
set  to  insure  optimum  labor  productivity  at  all  levels  or  deportments 
engaged  in  the  monufacturii^  process*  The  contractor  shall  use  this 
information  to  most  efficiently  monage  oikI  control  the  monufacturing 
departments*  The  system  procedures,  orKl  any  subsequent  chonges  shall 
be  coordinated  with  the  local  Government  representative  prior  to 
inplementation*"  (4)  : ‘ 

In  1972,  the  so-callcd  Sogamore  Study  provided  the  bosis  for  the  state- 
ment  that  even  at  Aircraft  1000,  45%  of  the  time  was  non-productive. (5) 
The  study  was  officially  publish^  os  the  Summary  of  Air  Force/Industry 
Manufacturing  Cost  Reduction  Study,  28  August-1  September  1972,  Air 
Force  Materials  Loborotory  Technical  Memorandum  AFML-TM-LT-73-1, 
January  1973*  Figure  3 below  is  developed  from  the  information  in 
that  study.  ^ v-  ^ 

There  are  several  interesting  inferences  that  can  be  mode  from  Figure  3* 
•The  first  concerns  inefficient  methods*  The  inference  is  simply  that 
there  Is  significant  potential  to  improve  productivity  by  methods  improve- 
ment, especiolly  by  carefully  defining  the  methods  right  from  tho  start. 

:•  By  doif^  that,  by  better  defining  methods  initially,  the  amounts  of  time 
consumed  by  famifiaiization  with  work,  and  crowdir>g  would  be 
drastically  reduced;  ir>efficient  job  ossignment  should  also  decrease. 

The  factors  that  affect  productive  time,  such  as  bad  parts,  inefficient 
methods,  and  crowding,  inefficient  job  assignment  are  primarily  manage- 
ment responsibilities  os  opposed  to  worker  or  operator  responsibilities* 

To  some  considerable  extent,  monogement  is  also  responsible  to  see 
that  training,  supervision,  and  clarity  of  work  instructions  minimize 
the  time  required  for  fomiliarizotion  with  the  job* 

Familiarization  with  the  job  is  representative  of  learning  — in  the 
context  of  learning  curves*  The  pherx>merK>m  of  learning  has  been 
described  mathemotically  in  terms  of  production  time  decreasing  by  a 
fixed  percentoge  os  production  quantities  double*  Aerospace  schedules 
arxl  budgets  traditionally  reflect  this  phenomenom*  Often  for 
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purposes  of  estimating,  the  time  to  moke  the  initial  production  units 
ore  calculated  by  figuring  that  the  erginecred  time  standards  or 
efficient  production  will  be  achieved  at  Unit  1000  or  5000  or  so. 

Then  the  estimate  for  the  early  production  units  are  created  by 
"backing  up"  the  learning  curve.  If  the  standord  was  to  be  assumed 
to  be  met  at  Unit  1000  and  on  84%  curve  were  used  (thot  is  os 
quontities  double,  the  unit  production  time  decreases  to  84%  of  the 
eorlier  units'  production  time),  then  the  initial  estimote  %vould  be 
about  5,7  times  the  thousandth  unit  or  the  taigct  for  efficient  production. 

In  reolity,  as  can  be  seen  from  Figure  3,  the  phenomenom  is  not  so  ^ 
much  one  of  learning  as  one  of  cost  improvement.  Production  efficiency 
Increases  more  as  a result  of  other  factors  than  os  a result  of  learning 
or  fomiliorizotion  with  vrork.  In  fact,  csfter  the  first  dozen  units  or 
so,  fomiliarizotion  with  work  does  rK>t  become  a significant  cause  of 
Inefficiency,  Therefore,  one  could  conclude  from  Rgure  3 that  an 
operator  does  not  hove  to  work  on  thousands  of  aircraft  before  there  Is 
hope  of  achieving  perfomnee  io  on  engineered  starxiard*  because,  by 
definition,  an  engineered  standard  is  the  time  it  should  take  a reosondbly 
skilled  operator  to  do  a job  he  is  familiar  with,  t 

Implementation  Efforts 

On  25  March  1973,  General  George  Brown,  who  wos  then  Commander 
of  Air  Force  Systems  Command,  chartered  Project  ACE  (for  Acquisition 
Cost  Evaluation).  (6)  One  of  the  results  of  the  project  was  the 
identification  of  an  opportunity  for  potentially  significant  savings  In 
direct  marxi factoring  Idbor  on  major  weopon  systems,  bosed  on  the 
Sagamore  Study,  Consequently,  Air  Force  Contract  AAonogement  Division 
wos  tasked  to  develop  a military  starxkird  for  work  measurement,  (7) 

The  standard  was  to  be  designed  to  require  disciplined  and  integrated 
work  meosurement  systems  as  a vehicle  to  achieving  reductions  in 
direct  manufacturing  labor  and  other  costs. 

The  original  AFCMD  effort,  doted  4 Jonuory  1974,  contained  the 
following  provisions  that  were  later  modified: 

Type  I (engineered)  standards  to  have  an  occuracy  of  +10%  with 
95%  confidence, 

A plan  to  provide  for  progress  toward  a gool  of  90%  coverage  of 
touch  (direct  monufacturing)  Idbor  by  Type  I standards. 
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Controctor  to  foke  full  odvontoge  of  the  stondord  lime  data 
ovoiloble  in  the  DOD  Dota  Bonk.  . • . 

• ^ IV  ! • “ . '*  ill*  ••  .»t  ^ *r 

The  Government  to  retain  rights  to  all  bosic  and  multipurpose 
standard  time  dato  developed  under  contracts  of  which  this  standard 
Is  a part.  (8)  . v..  •;  irt  n . 

Tfie  draft  standard  was  submitted  to  AFSC  by  AFCMO  and  subsequently 
submitted  to  industry  for  coordination.  This  established  a series  of 
. interactiofv  ultimotely  involving  AFCMD,  AFSC,  DOD,  and  professiorral 
and  industry  association  representatives.  The  ob{ective  was  to  produce 
o professionally  sound,  effective,  vrorkoble  standard  acceptoble  to  . 
both  the  Air  Force  and  industry.  i j .iMr;;03i.  tn-aciirS:^vM 

The  initial  industry  response  come  through  the  Council  of  Defense  ^ *c  ‘ 
ond  Space  Industry  AssMiatioro  (CODSIA).  That  response  is  * _ 1 .cl  - 

summorized  below: 

- % „ '.O’CM-  i.,.  ..  tVJ  -'tv 


'Appreciation  of  the  opportunity  to  review  the  proposed  standard. 

Concurrence  with  obfective  of  obtaining  maximum  productivity 
and  cost  effectiveness  while  fulfillir^g  contractual  requirements. 


Concern  that  the  standard  represented  orK>ther  layer  of  management 
disciplir>e  ond  control.  . . / ‘ i:. 

•\  _ * • . 1^-  ' 

Objections  t'xit  the  standard  was  vaguely  specified,  structurally 
complex,  extremely  burdensome  to  install  and 'maintain,  administratively 
expensive,  redundant  to  existing  contractually  required  management 
systems,  and  contrary  to  DOD  expressed  policy. 


Emphasis  that  the  proposed  standard  "would  infringe  on  a private 
compony's  right  to  manage  its  own  business  so  as  to  pose  a threat 
to  our  competitive  economic  system.  Moreover,  the  imposition  of 
^socialized*  standords  on  industry  would  have  the  effect  of  stifling 
competition  between  componies,  contrary  to  the  design  to  cost 
concepts  ond  value  engineering."  • < 

Expression  thot,  "the  government's  desire  for  reduced  cost  can 
best  be  attained  through  contractor  competition,  rx>t  government 
imposed  performance  measurement." 
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Contention  that  the  Project  ACE  indictment  of  ocrospoce  industry 
manufacturing  inefficiencies  wos  unwarranted. 

4 f r.  * r\ 

Summorization  that  manogement  techniques  must  be  determined  on  ^ 
Q case-by-cose  basis  and  are  best  left  to  the  discretion  of  irKlividual  . 
compony  manogement.  ‘ ‘ \ 

Emphasis  that  industry  currently  uses  work  measurement. 

Contention  that  current  controls  are  generally  odequote.  “ ^ 


Recommendation  that  the  stondard  rieed  rK)f  and  should  not  be 


issued.  (9) 

. - W3 

Need 

A new  need  was  created.  This  was  the  need  to  resolve  irxlustry  concern 
sufficiently  to  get  a military  standard  published.  ‘ • \ 

Research  , , 

• ■ * > <-  * *'  rrz..'  ' Ot 

Commercial  light  plone  manufacturers'  practices  and  those  of  the  aerospace 
irxlustry  were  reviewed."  We  literally  searched  again  through  old  studies 
arxi  records,  as  well  os  iriitioting  new  investigotions.  This  research  was 
designed  in  reality  to  reassure  ourselves  that  our  work  mode  sense. 


One  reassuring  criticism  received  was  that  the  proposed  military  starxlord 
represented  a textbook,  motherhood  approach  to  work  measurement.  It 
was  our  intention  that  the  military  standard  be  just  that,  and  more. 

The  "more"  was  that  it  must  be  implementoble.  " 

■-  q 

Implementation  Efforts 


Our  approach  was  to  promote  understanding.  To  that  end,  AFCMD 
porticipoted  in  and/or  fostered  the  following  activities: 


The  development  of  a Work  Measurement  Systems  Evaluation  Course 
to  teach  the  Government  representatives  in  plant  the  techniques  they 
would  need  to  administer  the  provisions  of  the  Work  Measurement  Military 
Standard  in  a contiact.  This  course  is  taught  by  the  Army  Monogement 
Engineering  Training  Agency  (AMETA).  Thus  far,  four  classes  of  obout 
twenty  each  have  been  taught. 
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A seminor  at  one  confraclor*s  rocilify  detailing  how  the  MIL-STD 
might  be  implemented  there. 

Presentations  ot  American  Institute  of  Industrial  Engineers  (AIIE) 
Chapter  meetings  in  Los  Angeles  and  Orange  County,  at  the  AIIE 
Aetospoce  Division  Notionol  Conference,  and  at  the  AIIE  National 
Conference  In  1977. 

Presentations  of  the  Spring  and  Fall  MTM  Conferences  in  1976  and 
ol  the  Sjpring  MTM  Conference  in  1977. 

Articles  on  work  measurement  ond  the  military  standard,  which 
opf>ear  in  lixkistrial  Engir^r,  The  Defe.de  Monogement  Journal^  and 
the  MTM  Joumol. 

AFCMD  membetship  In  the  MTM  Association. 

A program  for  MTM  training  and  certification  of  certain  Industrial 
engineers  who  are  resident  ot  Air  Force  Plont  Representative  Offices. 

After  revisions  aid  continued  dialog  witli  industry  and  professional 
association  people,  but  without  their  complete  agreement, MIL-STD-1567 
(USAF),  Work  Meosurement,  was  published  with  the  dote  of  30  June 
1975.  (10)  The  requirement  for  use  of  and  references  to  a DOD  Data 

Bonk  were  deleted.  The  requirement  for  government  data  rights  to 
standards  developed  in  response  to  the  military  standard  was  also 
deleted.  Additional  compromises  were  mode  in  the  areas  of  coveroge 
and  accuracy. 

The  standard  wos  developed  to  apply  to  major  Air  Force  weapon  system 
production  contracts  — those  of  $20  million  annuolly  or  $100  million 
total.  It  was  also  developed  to  apply  to  a full-scale  development 
contract  of  $100  million  or  more  that  was  io  precede  such  a major 
Air  Force  weapon  system  contract.  Coa traction,  focilities,  off-the- 
shelf  commodity,  time  and  materials,  reseoich,  study,  and  development 
contracts  other  than  those  precedir)g  a mojor  production  contract  were 
excluded.  (10)  The  standard  does  not  become  effective,  of  course, 
until  incorporated  into  controct. 

The  Intent  of  the  opplicability  provision  was,  of  course,  to  limit  the 
opplicotion  of  the  standard  to  those  instances  where  it  would  be 
clearly  beneficial. 
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The  standard  is  also  applicable  through  flow-down  to  certain  sub- 
controcts;  though  flow-down  may  be  waived.  (10) 


The  general  requireseents  of  the  stondord  are: 


^•:;T  Tr 


"a.  A work  measurement  plan  and  supporting  procedures. 


*h.  A cleor  designation  of  the  organization  end  personnel  respon- 
sBsility  for  the  execution  of  the  system. 


*c.  A plan  to  cstoblish  and  maintain  engineered  labor  slondords 
of  known  occurocy.  , .i 


*d.  A pirns  of  continued  improved  work  methods  In  connection  with 


the  established  labor  stemdard. 


*e.  A deRtted  plan  for  the  use  of  labor  stofKkirds  as  on  input  to 
budgetif^,  estimoting/  production  planning,  and  'touch  Idbor*  perfbimance 
evoluotion."  (10) 


The  intent  is  simply  for  the  controefor  to  hove  a documented,  disciplined. 
Integrated  week  measurement  system.  i a 


Certain  specific  requirements  of  the  standard  will  be  highlighted  below. 
The  first  of  these  b for  Type  I standards  to  "be  backed  up  by  sufficient 
data  to  statistically  support  an  occurocy  of  +25%,  with  at  least  a 90% 
confiderKe  level.”  (10)  This  represents  o significant  compromise  from 
the  original  position  of  +10%  with  a 95%  confidence. 


Why  an  accuracy  requirement  of  +25%  with  a 90%  confidence  level? 

Thb  is,  in  feet,  a maximum  compromise  position.  The  requirement  is 
a minimum  common  denomirKitor  that  should  be  exceeded  when  warranted. 
Industry  resistance  to  the  occurocy  requirement  resulted  in  revision 
downward  of  earlier  proposed  requirement  to  the  90%  + 25%  level. ^ 

There  is  still  some  industry  suggestion  that  90%  + 25%  os  a common 
dcrK>minQtor  b excessively  stringent.  Thb  oppeors  to  be  somewhat 
Irrational.  Most  standards  rouiinely  exceed  the  90%  + 25%  requirement. 
Most  standords  in  commercial  industries  without  incentives  probdbly 
exceed  95%  + 10%.  Using  time  studies,  35-1/2  times  fewer  observations 
ore  needed  to  demonstrate  90%  + 25%  as  to  demonstrate  95%  + 5%.  (11) 

The  appropriate  MTM  techniques*~noffnally  result  in  on  accuracy  of 
95%  + 5%. 


There  b also  a requirement  for  the  contractor  to  "develop  and  implement 
a Work  Measurement  Coverage  Plan  which  provides  a time-phased 
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schedule  for  ochieving  80%  coverage  of  oil  cotegories  of  touch  lobor 
by  Type  I stondords."  Furthermore,  this  plan  "sholl  be  based  on 
cost  trade-off  analyses  which  relate  savings  to  be  accrued  through 
improved  productivity  and  simplification  of  work  methods  to  the  cost 
of  attaining  Type  I standards  coverage."  (10) 

Why  80%  coverage?  There  ore  three  reasons.  The  first  is  thot  \ 
occotding  to  the  Pareto  Distribution  or  ABC  Curves,  we  could  expect 
20%  of  the  time  standards  to  cover  80%  of  the  direct  lobor  hours.  The 
second  s that  about  80%  coverage  of  the  direct  labor  hours  oppeors  to 
be  a reasonable  minimum  to  give  credibility  to  and  promote  confidence 
In  a work  measurement  program.  The  third  reason  is  that  20%  of  non- 
coverage  should  provide  suflicient  flexibility  by  o reasondble  margin  to 
accommodate  the  true  anomalies  that  may  not  justify  engineered  time 
startdards.  When  one  looks  at  the  coverage  achieved  in  the  commercial 
n^et,  20%  of  non-coverage  seems  Indefensibly  high.  However,  the 
8W  coverage  rec^irement  seems  a reasonable  compromise  from  the 
origirtol  90%  coverage  requiremeBt. 


The  si4>{ect  of  realization  foctcn  ond  their  use  Is  cornplex,  and  some 
discussion  b needed  to  understand  the  militory  standard  requirement  in 
that  area... 

• • • I S'  *•*■'*•,.*• 

Simply  slated,  standards  are  often  adjusted  by  a variety  of  factors  for 
a vcricty  of  uses  for  a variety  of  reasons.  The  result  of  thU  process 
can  promote  or  denigrate  effecthrertess.  If,  for  example,  one  estimates 
using  standards  modified  by  a reolizotlon  factor  bo.ed  on  experience, 
then  the  standards  con  become  irrelevant.  Philosophically  (but  not 
mathematically),  this  can  be  e>q>fe$sed  by: 


Estimates  = Standards  x Realization  Factor  = Sandords  x Actuals  = Actuals. 

Standards 


or 


ESTIMATES  = ACTUALS 


In  other  cases,  for  perfomrance  measurement,  targets  ore  created  by 
assuming  stondords  will  be  ochieved  (100%  efficiency)  at  specific  unit 
of  production,  soy  1000.  Thus,  os  discussed  earlier,  the  performance 
target  for  Unit  1 on  a 84%  curve  would  be  5.7  times  the  stemdard. 

In  some  systems,  even  these  targets  ore  adjusted  further  on  the  bosb 
of  past  performance  to  targets.  If  one  allocates  and  controls  the  budget 
bosed  on  these  targets,  then  monoging  to  budget  provides  little  Incentive 
for  efficiency.  (12).  When  standards  provide  the  basis  for  determining 
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$chedjles,  shop  loads,  and  manpower  levels;  some  faeforing  may  be 
necessary  to  provide  realism*  Performance,  dbsentec  rates,  ond 
other  considerations  must  be  mode.  In  these  instances,  the  use  of  ' ' ’ 
realization  factors  is  beneficial.  The  military  stondard  requires: 

“when  Idbor  standards  have  been  modified  by  realization  fdetors,  eoch 
element  ^*hich  contributed  to  the  total  factor  shall  be  identified. 

The  OTKilysis  supporting  each  element  will  be  ovailoble  to  the 
Government  for  review."  (10) 

• : * ; c'/ » sin • .1 

The  purpose  is  to  ossure  that  such  factors  are  correctly  used  in  esti- 
rfioting*  btimotes  should  rK)t  condone  or  perpetuate  post  inefficiencies. 
Simply,  we  wish  to  inhibit  inflated  estimates.  We  want  the  estimates 
to  stcrxlards  to  compare  fovordbly  with  the  budget  targets  so  tha:  managir^ 

to  budget  can  better  promote  efficiency.  . . , 

* . *4.-  I r ra  irZ^*rT 

The  military  standard  recognizes  that  manufacturing  labor  performs  to  ‘ 

expectations.  When  history  with  all  of  its  atterxlant  inefficiencies 
forms  the  basis  of  estimates,  these  estimates  when  priced  inftuerKe  the 
budget.  Of  course,  the  learning  curve  and  realization  factor  tnust  also 
be  considered  when  setting  planned  time  targets  for  touch  lobor  perform 
mar>ce  on  the  factory  floor.  What  results  then  is  a torget  which  can 
be  as  much  os  25  times  greater  than  the  time  actually  required  to  perform 
a given  task.  Thus,  if  an  employee  is  given  50  minutes  to  do  a task 
which  "should  take"  two  minutes,  peer  pressure  alone  will  dictate  that 
he  consume  the  allotted  time.  Unfortunately,  these  targets,  to  the 
extent  that  they  are  so  perceived,  become  the  expectotions  to  which 
mamifocturing  Idbor  performs,  otkI  the  cycle  is  self-perpetuating. 

When  standards  provide  the  basis  for  estimates,  then  inefficiencies  must 
be  SO  identified.  When  standords  similarly  provide  the  expectations 
for  performance,  unreasonably  poor  performance  con  be  less  comfortably 
tolerated.  Thus,  the  military  standard  works  through  both  ldx>r  perfor- 
monce  control  and  budgetary  control  mechanisms  to  help  assure  the 
expectations  that  determine  performance  are  reasonable  arid  represent 
recBortoble  efficiency. 

The  militory  standard  requires  an  audit  at  least  yeorly.  The  exact 
requirement  is: 

"The  contractor  shall  use  on  internal  review  process  to  monitor  the 
woHc  measurement  system.  This  process  shall  be  so  designed  that 
weaki>esses  or  failures  of  the  system  are  identified  and  brought  to  the 
attention  of  management  to  enoble  timely  corrective  action.  Written 


procedures  will  describe  the  oudit  techniques  to  be  used  in  evaluating 
-system  complimce.”  (10) 

Why  on  audit  requirement?  In  short,  to  help  assure  system  discipline. 

We,  the  Government,  are  rK)f  osking  to  audit  the  contractor.  We  are 
osking  him,  the  conlroctor,  to  review  himself.  We  view  audit  os  the 
key  to  system  discipline.  I have  heard  others  outside  the  Air  Force 
express  the  same  opinion.  Audit  is  the  key  to  confidence  in  the  system 
ond  the  time  standards.  Demonstrating  accuracy  and  coveroge  is  important, 
not  so  much  for  the  Air  Force  os  for  the  contractor.  If  the  contractor's 
work  force  doesn't  believe  in  the  standards,  the  value  of  those  staryiards 
will  be  largely  lost. 

. ' ..  T.  L-  . L 

The  audit  is  required  by  the  MIL*STD  to  be  of  specified  scope.  That 
scope  irKludes  oil  of  the  aspects  of  the  work  measurement  system  we 
thought  manogement  should  be  concerned  obout:  occuracy  ^ storvlords; 
validity  of  methods,  coverage;  effectiveness  of  starxkirds  used  in  planning, 
estimating,  budgetlr^,  and  scheduling;  timeliness,  occuracy,  ortd  trace- 
ability  of  production  count  reporting;  accuracy  of  performance  reports; 
reasonableness  of  efficiency,  gcxils;  and  the  effectiveness  of  corrective  actions 
resulting  from  variance  arxilyses. 

We  required  that  the  audit  reports  be  available  for  Government  review.  How 
else,  how  less  painfully  for  the  contractor  could  the  Government  assess  the 
disciplirm  of  the  contractor's  work  measurement  system?  Though  wc  hove 
leff  the  details  of  the  audit  to  the  contractor,  we  have  demonstrated  to 
our  own  satisfaction  that  employing  a technique  such  os  sequential  sampling 
mokes  auditing  coveroge  ond  occuracy  quite  practical  and  economical. 

Need 

With  the  publication  of  Ml L-STD-I567(USAF),  three  new  needs  orose. 

First,  the  MIL- STD  needed  to  be  incorporated  into  a weapon  system  contract. 
The  MIL- STD  is  effective  — thot  is,  is  a requirement  — only  after  it 
has  been  incorporated  into  a contract.  At  this  point  in  time,  we  hod 
devoted  an  enormous  effort,  the  result  of  which  was  a tool.  No  benefits 
had  occrued  or  would  accrue  until  the  tool  was  used  --  until  MIL- 
STD-I567(USAF)  was  placed  on  contract. 

That  rK>  benefits  had  yet  accrued  was  really  not  true  in  one  sense.  That 
sense  is  that  there  wos  an  irKreosed  awareness  on  the  part  of  irxlustry 
that  the  Air  Force  was  becoming  significantly  more  serious  otkI  more 
effective  in  odvocoting  MIL-STD-I567(USAF).  This  olone  helped  to 
influence  dt  least  one  controctor  to  revitalize  his  work  measurement 
system  coa^lstent  with  Ml  L-STD-1 567 (U SAF) . Other  controctors  appeared 
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to  develop  a greater  and  more  constructive  awareness  of  their  own 
produttivity  and  of  their  own  work  measurement  systems.  However,  this 
benefit**  could  not  be  token  for  granted  and  was  ultimately  dependent 
on  the  successful  contractual  implementation  of  MIL-STD*I567(USAF). 


.The  second  need  was  to  successfully  administer  o contract  once  MIL-STD* 
1567(USAF)  was  so  mcorporoted. 


The  third  need  was  to  document  or  to  be  able  to  document  that  the  admini- 
stration was  successful  to  colculote  the  actual  savings  ochieved. 

....  JiiUr 


Reseorch 


In  order  to  incorporate  the  standard  into  a controct,  we  had  to  be  able 
to  demonstrate  to  the  system  program  offices  thot  such  a contractual  change 
would  be  beneficiaU  We  looked  for  every  example  we  could  firul  to 
demonstrate  thot  discipliried  work  measurement  systems  provide  significant 
savings  or^J  significant  return  on  investments  Several  of  these  examples 
have  been  documented  as  AFCMD  Lessons  Learned.  (14)  we  prepared  a 
full  eight-hour  indoctrinotion  briefing  which  wos  given  to  the  Aeronautical 
Systems  Division  and  is  scheduled  to  be  given  to  the  Electronic  Systems 
Division  orxJ  to  the  Armament  Development  . Test  Center. 


We  worked  with  AMETA  to  develop  on  Intensive  two-week  course  in  evalua- 
ting contractor  work  irteosuremer^t  systems,  which  is  oriented  toword  odministering 
the  requirements  of  MIL-SrD-1567(ySAF)« 


( We  developed  a methodology  for  calculating  savings  attributable  to  MIL- 

STD-1567(USAF).  If  an  adequate  beselirm  (current  performance  to 
storxlards)  exists^  then  improvement  in  Idbor  performance  can  be  dollarized 
V to  establish  gross  savings.  The  net  savings  are  calculated  by  subtracting 

the  increased  costs  attributable  to  additional  work  measurement  and  methods 
personnel  • 

( 

“ If  the  baseline  itself  is  odjusted  — that  is,  if  the  standards  or  targets 

against  which  performance  was  measured  were  changed  --  then  corresponding 
( adjustments  would  be  mode  to  the  gross  savings  to  reflect  the  boseline 

changes.  If  no  reasonable  baseline  existed,  work  sampling  would  be  used 
^ to  develop  an  estimoted  baselir^e.  (15) 

Implementotion  Efforts 

/ 


{ 


We  continued  our  public  relations  and  educational  campaign.  We  worked 
individually  with  progrom  offices  os  well  os  conducting  general  indoctrination 


efforts.  We  also  worked  individually  with  contractor  personnel.  These 
efforts  generally  provided  a bosis  for  mutual  understanding  and  progress 
and  helped  to  resolve  or  overt  unne^esscry  standoffs  based  on  purely 
odvenaiy  postures.  ‘ 

We  tried  to  develop  public  and  professional  support  by  working  closely  r . ; 
with  professional  associations  such  as  the  AIIE  ond  he  MTM  Association. 

These  associotions  could  not  advocote  MIL-STD-1567(USAF)  on  o partisan  r 
basis,  of  course.^  They  did,  however,  provide  excellent  forums  to  discuss 
ond  explore  the  issues  in  a public  and  professional  manner. 

Peer  pressure  and  criticism  is  a powerful  forre.  By  subiecting  our  reasoning 
and  that  of  industry  and  certain  contractors  to  vhot  type  of  scrutiny,  the 
natural  refereeing  rapidly  invalldoted  superficial  end  specious  arguments.  ' 
Acceptonce  or  perceived  acceptance  of  ccrtc«n  Air  Force  position  stotements 
In  thot  arena  provided  material  for  others  to  use  in  advocating  MIL-STD- 
1567(USAF).  Thus  we  extended  the  number  of  effective  supporters.  As 
the  success  of  MIL* STD-1 567(USAF)  appeared  more  certain,  more  Air  ^ 

Force  odvocates  appeared.  Their  espousal  of  MIL-STD-1567(USAF)  helped 
to  better  assure  the  succeis,  foi  which  they  could  oppropriately  claim 
credit.  Without  developing  depth  and  breadth  of  support,  few  researchers 
appear  to  be  successful  in  Implementing  reseorch  that  Is  originally  resisted 
or  considered  controversial.  Those  thot  understand  and  espouse  the  imple- 
mentation of  research  deserve  co— credit  with  the  researchers  for  successes 
achieved. 

We  worked  with  AFSC  to  develop  guidelines  for  opplying  MlL-STD-1567 
(USAF)  to  existir^g  contracts.  (16)(17)  We  also  are  developing  more 
specific  guidelines  for  our  administrative  personnel  in  the  contractor 
plants.  A new  chopter  to  AFCMD  Manual  84—1,  Manufacturing  Operations, 
containing  these  guidelines  should  be  published  in  the  near  future. 

We  Insured  that  industrial  englneeis  from  our  Air  Force  Plant  Representative 
Offices  and  other  affected  personnel  attended  the  four  special  offerings 
of  the  AMETA  Evaluation  of  Contractor  Work  Measurement  System  Evaluation 
Course.  We  established  MTM-1  certification  training  for  our  personnel 
so  that  they  could  be  demonstrdbly  proficient  applicators  of  the  world's 
most  widely  accepted  predetermined  starxjard  time  system. 

How  the  Savings  Are  Achieved 


Before  discussing  the  results  to  date,  some  additional  background  information 
on  exactly  how  work  measurement  ond  methods  engineering  creates  savings 
is  in  order. 
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One  key  to  the  savings  created  by  o disciplined  work  measurement  system 
ii  methods.  Applying  engineered  standards  to  operotions  specifies  directly 
or  indirectly  how  the  operations  should  be  done.  Setting  standards  esto- 
blishes  methods.  Right  from  the  stort,  certain  efficiencies  are  engineered 
in  and  certain  inefficiencies  are  engineered  out.  , 

Engineered  time  standards  provide  insurance  in  two  ways.  First,  they  help 
ossure  that  someone  hos  figured  out  how  to  moke  whatever  it  is  thot  needs 
to  be  mode.  Second,  they  help  ossure  that  if  the  method  selected  does 
not  work  well,  this  will  be  identified  early.  In  both  instances,  the 
insurance  is  provided  by  the  original  baseline  both  the  meth^s  ond 
times  specified.  It  is  sometimes  argued  that  the  original  baseline  is  un- 
infsortont  so  long  as  improvement  is  measured  to  it.  Experience  has 
shown  that  it  is  more  true  that  if  the  baseline  is  too  high,  it  is  impossible 
to  recover  no  matter  how  fast  one  improves.  ^ 

Two  large  commercial  aircraft  profects  were  claimed  to  hove  sustained  a 
fantastic  rote  of  improvement  or  leeming.  This  suggests  that  the  aircroft 
were  produced  before  someone  really  figured  out  how  to  build  them.  Both 
these  aircraft  programs  and  their  producers  experienced  some  finoncial 
difficulty  attributable,  in  port,  to  the  costs  of  producing  the  aircraft. 

Setting  standards  sets  performance  targets.  Just  the  act  of  setting  targets 
improves  performance  ~ or  so  researchers  would  have  us  believe.  Of  couiW, 
the  more  accurote  the  forgets,  the  greater  the  confidence  of  the  workers  in 
their  realism,  the  better  the  performance.  Standards  can  often  provide  a 
basis  for  estimates  of  greoter  realism  and  confidence  than  other  methods. 

More  relidble  estimates  mean  budget  forgets  more  directly  related  to  per- 
formonce  targets.  Managing  to  budget  then  becomes  more  meaningful  in 
terms  of  a tool  to  improve  performance.  (12) 

The  analysis  of  variances  of  performance  to  stondords  helps  focus  manogement 
attention  on  those  problems  of  greotest  significance.  Supervision,  methods 
Improvement,  and  other  actions  con  then  address  those  problems  to  improve 
performance.  Once  ogoln,  methods  oppeor.  By  applying  the  discipline 
of  methods  improvement  to  those  operations  identified  os  most  needing 
ottention,  improvement  becomes  routine. 

An  oudit  progrom  disciplines  the  system.  This  ossures  that  when  methods 
ore  used  other  than  those  on  which  the  stondord  is  based,  one  of  two 
things  happens.  In  one  cose  the  method  is  less  efficient  than  that  on 
which  the  standard  is  based;  the  standard  method  is  enforced.  In  the 
other  case  the  method  is  more  efficient;  the  standard  and  job  instructions 
are  revised  to  cepture  the  improvements. 
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Th^ie  are,  of  course,  many  other  benefits  which  result  from  a disciplined 
wofic  meosurement  system.  In  the  areas  of  planning,  scheduling,  loading, 
and  manpower  forecasting;  knowing  how  long  the  work  should  take  is 
invaluable.  Without  this  information  to  some  reasonable  degree  of  accuracy, 
several  undesirable  things  often  happen.  In  one  ccse,  the  estimates  are 
overly  conservative.  Too  many  people  have  loo  much  time  to  do  too  little. 
Thot  not  only  costs  too  much,  it  erodes  the  obility  to  perform  end  to  control 
performance.  Taking  the  time  allowed,  or  in  the  "Parkinsonian"  sense; 
peraiitting  the  work  to  expand  to  fill  the  time  available,  (13)  estdblishes 
new  and  expensive  norms  of  acceptoble  performance.  Btimates  far 
future  efforts  based  on  the  history  thus  created  compound  the  problem. 

Sometimes,  inefficiency  is  created  by  too  many  assigned 'to  do  a set  of 
tasb  and  getting  in  each  other's  way.  In  certain  cases,  this  causes  the 
contractor  to  foil  behind  schedule.  Ironically,  the  common  resportse  of 
putting  more  people  on  the  job  often  exacerb^es  the  problem.  Once  again, 
the  methods  approach  integral  to  work  measurement  would  both  help  avoid 
the  situation  in  the  first  place  md  result  i»  tetJucing  people  gettii^  in  each 
other's  way  in  the  second  place. 

The  use  of  engineered  standards  of  known  accuracy  to  establish  performance 
expectations  increases  the  reliobility  of  those  expectations.  Such  standards 
are  recognized  by  both  management  and  l<ix>r  as  representing  reasondbly 
attoiiKble  performance  with  reasonable  efficiency.  This  narrows  the  range 
of  performance  that  can  be  comfortably  tolerated  by  both  management  and 
lobor.  The  end  result  is  productivity  improvement  and  a better  chance  of 
meeting  schedules. 

In  another  case,  schedule  targets  may  be  unintentionally  inadequate.  This 
con  set  up  a chain  of  actions.  Normally  some  precautionary  pad  is 
included,  this  pod  is  then  removed  a step  at  a time.  In  some  coses, 
however,  by  the  time  it  is  discovered  thot  the  pod  was  inadequate  in  the 
first  place.  It  is  often  too  late  to  take  the  corrective  actions  that  would 
have  been  most  effective. 


One  should  not  infer  that  work  measurement  n a panacea.  It  is,  however, 
a proven,  established  technicfie.  A disciplined  work  measurement  program 
is  the  heart  of  effective  work  planning  and  control.  Work  plarming  and 
control  con  be  effective  without  work  measurement  and  con  be  ineffective 
with  work  measurement;  however,  experience  shows  the  opposite  is  over- 
whelmirsgly  more  often  the  case. 


One  other  key  to  savings  should  olso  be  discussed.  That  key  is  discipline. 
Most  of  the  savings  in  the  aerospace  industry  will  come  not  so  much  from 
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deyeloping  complete!/  new  work  measurement  systems,  but  from 
disciplining  or  revitalizing  existing  systems. 

• -.i  jrT 

The  militory  standard  fs  a management  tool  which  says,  in  essence,  we 
the  Government  have  been  generally  disappointed  with  the  self-discipline 
controcton  have  exercised  so  for.  We  expect  more.  We  deserve  better. 
We  ore  going  to  require  it. 

ir  , Vr-'-'  1.  t,- 

'•  We  have  noted  that  os  our  expectations  are  raised,  the  contractors  tend 
to  raise  their  expectations.  It  it  truly  the  increased  contractor  expectations 
and  discipline  that  create  the  savings.  The  results  which  I will  describe 
shorty  did  not  require  entirely  new  work  measurement  systems  to  be  created. 

The  billion  dollar  savings  potential  origirxilly  envisioned  was  bosed  on  a 
dollortzed,  twenty  percent  Improvement  in  direct  Ioii>or  productivity  spread 
across  Air  Force  contracts.  Since  thot  time,  some  more  precise  and  more 
supportciile  calculations  reinforce  the  billion  dollor  savings  potential  in 
another  manner.  . - v*  ^ :i 

• ' ' ' • .• 

The  base  ogaimt  which  MIL-STD-15£7(USAF)  was  designed  to  apply  was 

($100  million  or  more)  Air  Force  full-scale  development  and  produc- 
tion contracts.  The  face  value  of  major  AFSC  contracts  is  obout  $38 
billion.  Of  these  major  contracts,  the  vast  mojorlty  are  for  production 
or  foll-scole  development  efforts,  certainly  $25  billion  or  mere. 

Of  this  $25  billion,  more  than  30%  represents  the  burdened  value  of 
direct  monufocturing.  Positive  influence  on  at  least  two-thirds  of  this 
direct  manufacturing  effort  seems  feasible.  A twenty  percent  savings  or 
improvement  potential  still  appears  reosonoble.  This  would  create  a savings 
of  ($25B  X .20  X ,20)  — at  lecst  $1  billion.  Realistically,  the  savings 
potential  should  be  significantly  larger. 


In  addition,  cbout  40%  of  the  face  values  are  used  for  procurement. 

About  a five  percent  savings  should  be  mode  in  these  expenditures.  The 
r^on  for  those  savings  ore' the  flow  down  of  increosed  expectation  and 
discipline.  As  the  contractor  disciplines  his  own  work  measurement  system, 
he  will  expect  more  and  be  able  to  more  accurately  determine  what  manu- 
focturi^  should  cost.  This  will  result  in  his  improving  his  pricing  ability 
with  his  subcontractors  and  suppliers.  In  addition,  as  the  volue  of  MIL- 
5TD”J5i7(USAF)  is  impressed  upon  the  primes,  the  primes  moy  choose  to 
flow  down  the  requirement  to  their  subcontractors. 

This  5%  savings,  which  is  somewhat  understated,  would  total  ($25B  x.40  x.05) 
an  odditionol  $1/2  billion. 
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Results  to  Dote 


I -P  > 


To  dote,  MtL~STD-1567(USAF),  or  a modified  version,  hos  been  incor- 
porated into  four  contracts*  These  ore  the  B-1  Airframe  contract,  the  j 
B-1  Avionics  contract,  and  the  two  30mm  Ammunition  contracts  in 
support  of  the  A-10  Program.  ..  . ^ ^ 


At  one  contractor's  facility,  $6  million  in  savings  ossocioted  with  voluntary 
compliance  were  achieved  in  1976.  Savings,  of  course,  continue  to  accnje. 

l-'f  * Jir'f.  • I ,*  ‘ n'  • * • ' 7\  1* 

Voluntary  efforts  to  improve  or  enhance  existing  work  measurement  systems, 
sparked  by  MIL-STD-1567(USAF),  ore  resulting  in  improved  productivity 
and  corresporKling  savings  at  several  additional  contractors. 

*>  ^ #-•  |r  .J  . ' ' * • *»  »■  •.-Tj 

Incorporation  of  the  MIL-STD-1567(USAF)  or  modified  versicns  of  the 
stondard  ore  likely  for  several  additional  contracts  in  the  neor  future* 

» , f j ' . • ^ ..  • • V* 

The  total  savings  achieved  to  dote  approximate  $10  million  or  dbout  1%  > 

of  the  billion  dollar  potential*  However,  these  savings,  the  research, 
and  the  implementation  efforts  demonstrate  that  the  billion  dollar  potentiol 
exists  — At4D  that  with  peiseverence,  it  can  be  realized.  2,  ^ 
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N ^ Number  of  reodings  token  in  the  somple 


^-$2'  .v'**.  The  number  of  stcndard  deviations  required  based' 
,-((iv*t3vbci't  upon  the  confidence  level  established  by  management. 

• vfT  , 

S * Percent  accuracy  (expressed  as  a decimal)  required 
' estcblished  by.  manogement*  ' ..  i 

X ^ ■ Eoch  elementol  time  value* 

> • I •rijcr 

lor  95 5 E 1.960  .05  “ Vv  . 

' For  90  7 25  E « 1.645  . S « -25  ‘ j: 

X The  ratio  of  N*  95  + 5 to  N*  90  + 25  is  35.49.  ' 
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John  H.  Richardson 
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Speaking  for  pyself  and  for  the  gentleaen  accoapanylng  ae  on  this  platfom,  . 
we  appreciate  very  auch  the  oppertunity  to  contribute  soae  of  Industry's  views 
and  perceptions  to  this  syaposiai.  Since  ailltary  procureaent  Inherently  Involves 
OoO  contracting  with  Industry,  the  translation  of  procureaent  research  Into  action 
Is  necessarily  a Joint  concern  of  both  OoO  and  Industry.  colleagues  and  I will 
be  looking  at  a nuaber  of  facets  of  the  allltcry  procureaent  process  where  1iN>roveaents 
are  being  aade  or  Innovations  are  being  tried* 


I will  start  off  with  a Tiawery  af  several  systeas  acquisition  problea  areas  In 
which  Joint  attention  by  the  services  and  Industry  Is  yielding  soae  real  progress. 

Toa  Pownall,  Executive  Vice  President  of  the  Nartin-Narietta  Corporation,  will  then 
talk  about  the  particular  area  of  R&O  acquisition.  Finally,  Harry  Gray,  Chalraar  and 
President  of  UTC,  will  describe  soae  of  the  tangible  results  of  procureaent  research 
as  seen  In  the  UTTAS  program. 


There  ar»  a needier  of  channels  through  which  ailltary  and  Industry  aanagers  have 
worked  together  on  acquisition  Issues.  For  exaaple,  there  was  DARCOR's  Atlanta  IV  last 
month.  The  Atlanta  meetings  have  stimulated  face-to-face  discussions  on  the  acquisition 
process  between  senior  Aray  aanageaent  people  and  their  counterparts  In  Industry,  and 
have  been  an  outstanding  success. 


Then  there  a*^  the  Industry  association  and  professional  society  workshops, 
symposia,  and  seminars  that  concentrate  on  specialized  phases  of  the  acquisition  process 


And  this  year  this  annual  DoO  Procureaent  Research  Syaposlua  Includes  Industry  as 
well  as  government  presentations. 


I am  going  to  concentrate  on  a particular  series  of  events — Instigated  by  the 
military-— which  have  created  a healthy  cllaate  for  yet  another  approach  to  laproving 
systems  acquisition.  I refer  to  a series  of  small  top  level  "tell  It  like  It  Is* 
meetings  through  which  significant  problems  have  been  Identified  and  acted  upon.  These 
meetings  got  started  back  In  aid-1975,  when  General  Samuel  C.  Phillips,  then  connander 
of  AFSC,  wrote: 


•The  Air  Force  Systems  Connand  wants  to  explore  ways  of  Improving  the  acquisition 
process  and  strengthening  understanding  between  AFSC  and  ou.*  prime  contractors...  The 
objective  Is  to  provide  a two-way  flow  of  communication  which  cultivates  a *te1l  It  like 
It  Is*  environment,  focusing  on  the  major  Issues  In  the  acquisition  process. 


•There  1$  no  foraal  agenda.  All  discussions  are  on  a non -attribution  basis,  and 
the  size  of  each  group  Is  limited  to  or.e  representative  from  each  of  four  contractors. • 
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*IlJ***  General  Hilliaa  J.  Evans,  lAo  by  then  had  ass«aed  caanand  of 

AFSC,  gave  his  whole-hearted  endorsenent  to  continoation  of  the  prograi.  And  last 

®*"**‘*'  *•  Oeane,  Jr.,  Connander  of  the  Arny  Materiel  Oevelopnent  and 

Readiness  Conwnd,  Issued  a slallar  Invitation. 

u.^^***  "**'!.*«  ■**^*"55  taking  place  over  the  past  couple 

**  ^ assesswnt,  and  I believe  that  of  other  reprcsenmives  of 

‘•***  “etings  have  served  a useful  purpose 
things  been  accan«>11shed  as  a result  of  thenu  Here  Is  a sumary  of  sov  of  M 
problatt^iaentlfled  a^  discussed,  together  with  the  actions  taken  by  Air  Force  and 
Atmx  MnageMnt.  He'll  look  at  thew  one  by  one. 

BEST  AND  FINAL 

' df  i ' . ' - 

Problfli  \ - 


The  Best  and  Final  Offer  (BAFO)  technique  results  In  technical  transfusion,  buy-ins. 
and  auctloiiMrlng.  * ^ ' 


Air  Force  Action 


AFSC  directives  have  been  published  which  prohibit  repeated  calls  for  BAFO's. 


RepresenUtlve  procurenents  being  Identified  at  SANSO.  ASO,  and  ESO,  are  to  be 
negotiated  without  use  of  parallel  negotiations,  e.  g.,  negotiate  only  with  selected 
sourc6. 

Cost  realise  will  be  stressed  In  regulations  and  during  procuraKnt  conferences 
so  as  to  encourage  the  finding  of  non-responsiveness  to  unrealistically  low  bids. 

Arny  Action 


General  Deane  recently  reiterated  his  concent  trit-h  Best  and  Final  procedures  to  ( 

the  Un^  SecreUry:  He  has  reconaended  General  Officer  level,  by  dmc.  approval  for 
•tore  than  an  Initial  request  for  a Best  ano  Final  Offer.  Additionally.  OARCOM  head- 
quarten  wst  be  advised  of  any  repetitive  use  In  procurenents  of  $100,000  or  nore: 
and,  the  use  of  the  ASO  (lU.)  approved  new  four-step  procedure  Is  now  encouraged.  ( 

COST-TYPE  CONTRACT  WITH  CEILING 


Problem 


The  use  of  cost-type  contracts  with  ceilings  is  not  desirable  since  such  contracts 
entail  the  risk  associated  with  a fixed  ptice  contract,  but  have  all  the  controls 
Inherent  In  cost-type  contracts. 

Air  Force  Action 


HeaApurters  AFSC  corwhicted  a survey  of  field  offices  to  determine  the  extent  to  ( ^ 

which  this  contracting  technique  was  being  used.  Initial  results  indicated  a need  for  _ 

specific  policy  formulation  In  this  area.  Policy  taplementatlon  Is  being  studied.  ~ 


Amy  Action 


U 
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pie  Arv  Also  comiucted  a survey  of  use  of  this  contracting  technique,  and  as  a 

activities.  This  policy  indicates 
ceilings  will  not  be  Included  In  new  cost-type  contracts  or  through 
■edifications  of  r.lsting  cost-type  contracts  except  In  unusual  clrcuasUnces.  «1 
proposals  to  Include  such  ceilings  vst  be  referred  to  Headquarters  IMRCQN  for  review. 


TAILORINS 

Problew 


Over-application  of  “specs  and  standards*  results  In  unnecessary  costs.  There  Is 
need  to  provide  contractors  greeter  flexibility  to  offer  cost/perfomance  trades. 


Air  Force  Action 


DPP  Industry  Input  prior  to  finalization  of 

RFP,  and  RFP  clauses  (Including  Incentive  provisions)  have  been  developed  to  encouraoe 
contractor  coments  In  proposals.  encourage 

HandbMks  covering  selection,  tallorino.  and  application  of  specifications  have 

In  SAHSO  (6B-1)  and  at  Dayton  (ASOP  800-1),  and  the 


^ distributed  to  SPO  personnel  In  SAHSO  o««-ij.  «na  uw 

Air  Force  Institute  of  Technology  has  developed  a video  tape  training  flla  on  tailoring. 
A la^-scale  in-house  educational  effort  Is  being  undertaken  through  fomal  course 
creation  as  well  a^on-the-Job  training  to  encourage  wider  use  of  comnerclal  standards. 

'■  •'  '•  Amy  Action 

HICOM  has  released  a policy  letter  (700-3)  and  a regulation  (1-36)  requiring  that 
obtain  specs  and  standards  be  tailored,  reviewed,  and  approved  prior  to  application  on 
Rrrs  and  contracts. 

The  P^ran  Kanagenent  Course  at  Ft  Belvoir  has  been  expanded  to  Include  speclflca- 
tlon  and  standard  tailoring  and  application  of  specs  and  standards  In  the  Systen  X 
sinulation  exercise.  j *««  « 

And  provisions  have  been  made  for  a two-week  course  It  military  specification  and 
2^ag2nrSc^"it  Ft'Cee*  *****  industry  personnel,  at  the  Amy  Logistics 

That’s  a lot  of  action,  but  just  to  show  you  that  It’s  tough  to  win  then  all.  here 
are  a couple  of  areas  that  haven’t  been  moving  forward  rapidly.  First,  let’s  look  at... 

PCEP  (CONTRACTOR  KAWAGDCKT  SYSTEM  EVALUATION  PROGRAHl  . 

Problem 

..  . iJ"  situations,  full  application  of  CMSEP  entails  much  cost-ineffective 

detail.  At  least  at  those  facilities  where  there  is  a large  amount  of  coonercial  work, 
or  when  a contractor  is  performing  well,  the  application  of  CMSEP  should  be  reduced  or 
eliminated. 
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Air  Force  Action 

An  Air  Force  letter  In  January  1976  stated:  "A  review  of  this  area  Indicates 
that  CHSEP  Is  Intended  to  be  a self-tailoring  evaluation  tool  and  that  the  configuration 
and  breadth  of  a contractor's  nanagenent  systen,  as  It  applies  to  Governnent  prograns. 
deteralnes  the  degree  to  which  CMSEP  Is  applied.  AFCHD  controls  CNSEP  l^ilenentatlon 
by  using  a checklist  during  nomal  staff  visits  and  holds  quarterly  reviews  of  AFPRO 
operations.  In  addition,  tea*  audits  of  AFPRO  InplenenUtlon  of  CMSEP  will  begin  on 
26  February  1976."  So  far,  however,  we  In  Industi^  have  seen  no  appreciable  change  la 
the  application  of  CHSEP. 

Then  there  Is...  v ^ , 


CONTRACTOR  PAST  PERFORHAMCE 

Problen 


The  services  do  not  really  take  Into  account  a contractor's  past  performance  In 
the  source-selection  process.  We  believe  that  they  could  and  should. 

Army  Action  ; 

On  this  problem.  General  Deane  Indicated  In  January  of  this  year  that  he  saw  no 
practical  path  to  Improvement:  “The  OSO  study  group  on  Profit  '76  considered  tne  use 
of  past  performance  as  an  evaluation  factor  and  couldn't  get  anywhere.  The  concept  Is 
unbelievably  difficult  to  Implement  and  there  does  not  appear  to  be  any  ready  formal 
solution. - 

Mow  let's  get  back  to  the  positive  side  where  things  are  really  happening. 

VALUE  ENGINEERING  . , . 

Problem 

There  Is  much  interest  In  the  Value  Engineering  Program  at  the  higher  management 
levels  In  the  Air  Force,  but  at  the  working  level  there  is  considerable  lethargy. 

Air  Force  Action 

In  1976,  AFSC  issued  a regulation  requiring  the  setting  of  VE  goals,  a plan  for 
achieving  them,  and  regular  status  reports  to  AFSC  Headquarters  on  all  major  programs. 

In  October  1976,  a V.  E.  workshop  was  conducted  with  the  AFSC  Division  staff  value 
engineers  to  assess  problems  and  to  structure  a positive  V.  E.  enhancement  program.  In 
January  of  1977,  AFSC  sponsored  a “V.  E.  Congress*,  with  participants  from  AFSC  System 
Project  Offices,  Air  Force  Plant  Representative  Offices,  other  DoD  organizations  and 
contractors. 

AFSC  has  scheduled  top-level  briefings  this  spring  for  all  subordinate  CDwnands. 
The  briefings  will  outline  tiie  VE  program,  describe  Its  advantages,  and  request  commit- 
ment to  achieve  results.  In  addition,  the  Air  Force  Institute  of  Technology  Is  con- 
ducting VE  training  programs  for  management  and  contracts  people  at  all  commands. 
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Incidentally,  several  changes  are  In  the  process  of  being  nade  to  the  Amed  Services' 
Procurenent  Regulations  VE  sections,  which  should  reduce  their  conplexlty,  clear  up  the 
language,  and  encourage  broader  application  by  DoO  agencies  and  Industry. 

. -t'’ 

c/sese  (C0ST/SO1EDULE  COfTWOl  STSTEWS  CRITERIA)  ) i 

' - ’.f  -^03 

Problea  ’ - i „ . i 

.1  - , , V J •,  ‘ rh  h ’ll 

The  need  for  C/SCSC  on  nature  prograns  should  be  reviewed.  After  several  years  of  . 
a production  progran,  the  cost  effectiveness  of  C/SCSC  Is  questionable. 


Air  Force  Action 


. • i4  f 


% Headquarters  AFSC  detemln^rd  that  re-evaluating  and  appropriately  nodifying  cost 
reporting  requirenents  as  a progran  natures,  has  nerlt.  AFSC  product  division  procure- 
nent  organizations  were  advised  of  the  need  for  progrwi  offices  to  continually  review 
the  stablllty/naturlty  of  their  programs  and  to  "tailor*  cost  perfomance  reporting 
requirenents  accordingly. 


Tri-Service  Action 


. Three  months  ago.  Industry  received  a letter  from  the  Tri -Service  Perfonnance 
Neasurenent  Joint  Executive  Group  which  said.  In  part:  * The  purpose  of  this  letter  Is 
to  request  the  recomaendatlons  of  your  firm  regarding  possible  Improvements  In  the 
application  of  C/SCSC  to  the  management  control  of  production  contracts.  The  PHJE6  Is 
considering  the  development  of  a modified  C/SCSC  Interpretation  exclusively  for  production 
contracts.  It  Is  also  contemplated  that  a committee  representing  contractors  will  be 
organized  to  work  with  the  PMJEG  In  this  effort.* 


L i fGr 


. ?•'  •* 

i.v: 


Now  In  addition  to  the  problems  I have  Just  touched  on,  there  are  of  course  many 
other  areas  in  which  improvement  or  Innovation  Is  called  for.  Here,  for  example,  are 
three  subjects  that  have  not  yet  received  a thorough  airing  at  our  joint  meetings,  and 
that  we  shall  be  discussing  at  future  sessions: 


Planning  Purpose  Prices  for  Interesting  Qpttens 


Buyers,  RFP  writers;  and  program  managers  should  be  stopped  from  asking  for 
%ihat-1f*  prices  for  options  that  may  be,  but  are  not  now,  a part  of  the  program  to  be 
priced  In  a source  selection.  Such  a process.  If  permitted,  inescapably  leads  to  ”11ar*s 
poker.* 


1 


Front-end  Loading  to  Reduce  Life  Cycle  Costs 


Nuch  has  been  said  about  paying  more  for  Initial  acquisition  If  It  will  save 
support  costs  In  the  out  years.  These  support  costs  used  to  account  for  less  than  SO 
per  cent  of  total  life  cycle  costs.  That  figure  Is  now  up  to  about  65  per  cent.  The 
principle  of  paying  now  to  save  later  makes  sense,  but  does  OoD  really  mean  it  and  how 
can  we  implement  the  principle? 
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Funding  Ccaoetitive  Prototyping 
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. I M the  first  to  agree  that  the  Govemaent  has  really  been  getting  aore  than  Its 
■oncy's  north  fro*  coepetltlve  prototyping.  I also  agree  that  OoO  has  beoae  gulte  ' 
proficient  at  estlaatlng.  However,  we  ought  to  be  able  to  recognize  on  the  basis  of 
experience  that  It  Is  not  In  the  best  Interest  of  OoO  or  Industry  to  place  a contractor 
in  a position  of  great  financial  risk,  and  that  Is  what  these  underfunded  prototype 
prograas  do.  If  we  are  going  to  continue  with  coapetitive  prototyping,  reasonable 
funding  levels  should  be  provided  to  the  contractor  participants. 

it  ■ W.W;V.  . ,-v  • . (,< 

In  suanary.  I have  presented  soae  concrete  exa^>1es  of  how  Industry  and  goveniwnt 
are  working — hard  and  together — to  laprove  our  understanding  of  each  other's  probleas 
and  to  Inwove  the  acquisition  process.  Significant  things  have  been  acca^>11shed; 
genuine  progress  Is  being  node.  Procurcaent  research  Is  being  translated  Into  action.  | ’ 
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Good  afternoon  Ladles  and  Gefitlenen.  " f- 

What  I do  Intend  to  talk  to  you  about  Is  two  or  three  RSO  eicnents  and  how  they  relate 
to  the  A10G  cirailar  about  Independent  research  and  development.  That  last  subject  Is  very 
near  and  dear  to  all  of  us  on  this  panel  and  all  of  us  In  Industry. 

Research  and  Development  consists  basically  of  the  technology  base  and  enhancement, 
which  Is  simply  the  early  pursuit  of  Ideas,  concepts  and  facts,  and  the  full  scale  develop- 
ment programs.  The  first  par^t  accounts  for  a couple  of  billion  dollars  of  the  total  ROITE. 
but  1t*s  composed  of  about  20.000  separate  efforts  and  that's  a veiy  substantial  nimter. 

It  would  be  substantially  larger  and  a good  deal  less  nanagable  were  It  not  for  the  indepen- 
dent research  and  development  effort  about  which  I will  speak  more  fully.  The  full  scale 
developments  account  for  about  8 billion  or  70X  of  the  overall  R0TI£  effort.  In  this  area, 
significant  changes  In  the  major  systes  acquisition  process  will  be  occuring  as  a result  of 
(M  circular  A109.  which  Incorporates  several  recommendations  of  the  Coorisslon  of  Government 
Procurement. 

There  will  be  an  lipact  on  R&O  acquisition  also.  Following  the  Agency  Head's  approval 
of  a Mission  Need  Statement.  Industry  Is  to  be  sollcitated  for  It's  ideas  of  system  designs 
concepts  that  satisfy  the  mission  need.  Uhat  Is  being  attempted  here  Is  to  start  from  a 
general  area,  and  work  all  the  way  through  the  system,  to  arrive  at  a matching  of  that 
need  with  some  hardware,  system  or  at  least,  an  answer.  1 believe  that  Is  an  attract- 
ive proposition.  The  solicitation  Is  to  explain  the  need  In  mission  or  capability  ter«. 
not  equipment  terms,  and  It  Is  to  give  schedule  objectives,  constraints  and  proqram  object- 
ives. This  contrasts  sharply  with  the  traditional  practice  of  delineating  a major  system 
In  explicit  ter«  and  soliciting  Industry  for  the  conduct  of  the  R&O  to  evolve  the  specific 
system  described.  A109  enables  contractors  to  be  free  to  propose  their  Oh^  technical 
approach,  main  design  features,  sub-systems,  alternatives  to  schedule,  cost  and  even  capabil- 
ity goal,  competitive  parallel  short-term  contracts  are  to  be  used  to  fimd  well  defined 
activities  during  exploration  of  system  design  and  concept  alternatives.  The  superior 
concepts  are  then  to  be  funded  for  competitive  demonstration,  and  from  these  demonstrations, 
winners  will  be  selected  for  full  scale  development  and  potentially,  the  Initial  production. 

What  arc  the  Inpllcatlons  for  Industry?  For  Industry,  there  will  be  a very  considerable 
e^>has1s  on  the  front  end  of  the  procurement  process  with  the  attendent  requirements  to 
explore  Innovative  concepts  addressing  customer  needs.  This  In  turn  will  require  that 
Industry  devote  more  of  Its  Indepmdent  research  and  development,  the  IRAQ  piece,  to  the 
exploration  of  concept  and  feasibility  stuqy. 


IR&O  Is  broddly  defined  as  that  MD  not  required  by  a contract  or  a grant.  The  major 
defense  contractors  negotiate  annual  agreements  with  the  000  based  on  submission  of  exten- 
sive technical  plans  describing  In  detail  the  IR&O  they  Intend  to  conduct  during  the  year. 
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These  annual  agreements  define  a dollar  ceiTIng  representlrr  ee  maximum  mount  of  a 
contractor's  IR&D  program  that  000  will  accept.  Almost  everybody  In  Indbjtry  spends  well 
over  that  celling  allocation,  and  It  can  vary  from  a few  percentage  points  to  maybe  as  much 
as  30  or  40S  depmdlng  upon  the  level  of  activity  of  tdilch  they're  engaged.  We  can't  control 
that  because  we  might  be  very  active  In  one  six  month  period,  and  mayte  not  very  active  In 
the  next  six  montlis.  Or  we  may  be  very  active  for  a two  year  period,  and  not  very  active  tra 
one  year  period.  So  we've  got  to  do  what  we've  got  to  do  when  we've  got  to  do  It,  whether 
the  money  has  been  provided  as  a piece  of  the  IlUO  allocation  or  not. 

ftr.  Curry,  who  Is  no  longer  In  Government,  described  IRU  this  way.  He  said  the 
Department  of  Defense  Is  coamitted  to  oompetltlon  for  Its  selection  of  suppliers.  This 
policy  obviously  Involves  the  cost  of  enabling  a sufficient  nuvber  of  suppliers  to  qualify 
themselves  to  compete  for  our  business.  The  effort  a cent  vetor  puts  Into  qualifying  himself 
IS  csiled  Independent  Research  and  Oevelopnent.  He  said  our  reasons  for  accepting  this 
cost  are  based  on  the  conviction  that  the  returns  of  coiS)etltlon  are  lonensely  greater  thm 
the  ejg)cndlture  It  Involves.  Coi^tltlon  loivers  the  direct  cost  of  our  requisitions.  He 
said  that  the  tasks  that  add  up  to  the  supplier's  cost  of  quailing  hinself  for  conpetitlon 
are  Internally  defined  by  the  supplier. Directions  fro*  000  that  classify  Usks  for  supplier 
cost  Mould  only  turn  conpetitlon  into  directed  procureaent.  Flnally»  he  Ment  on  to  s^y  that 
the  concept  of  Independence  Is  an  Inherent  feature  of  any  contractor's  IRSOm  Successful 
conpetitlon  (being  In  business)  Is  based  on  the  capabilities  one  Mill  possess  and  can  apply 
at  each  conpetitive  point. 

(bily  the  Individual  cospany  Is  accountable  for  the  decision  as  to  hoM  and  idrere  Its 
competitive  capabilities  nust  be  sustained.  This  Is  the  area  In  which  managenent  must 
use  Its  very  best  Judgennt  because  funds  111  spent  In  the  Independent  area  will  absolutely 
push  It  out  of  business.  If  they're  spent  properly,  then  whether  or  not  we  gain  the 
business*  we  are  at  least  conpetitive  In  the  right  area  and  we  have  an  opportunity  to  offer 
ourselves  as  reasonable  and  decent  competitors.  Othendse  we  become  nobodies  and  shortly 
after  that  we  just  disappear  from  the  scene. 

Mhat  Is  the  Impact  of  A109  on  IR&O?  Hell  the  longer  tem  affect  of  A109  will  be  a 
shift  In  the  reflective  proportions  of  a contractor's  conceptual  studies,  research  and 
development  categories*  which  make  up  his  IRAQ  program.  This  differs  with  the  highly 
optimistic  view  of  one  witness  to  the  Congressional  Hearing  on  IRAQ  In  Sept  75  who  ventured 
the  opinion  that  A109  type  procedures  would  eliminate  the  D element  of  IRAO  and  thus  result 
In  a reduction  of  90X  In  IRAO  costs.  Well  It  would  be  absolutely  delightful  If  It  were 
true  butit'snot.  As  I mentioned  earlier,  we  spend  substantial  amounts  of  our  money*  and 
, we  don't  make  any  money  on  It.  There  Is  no  money  to  be  made*  fundamentally*  In  the  RAO 
area.  As  a matter  of  fact,  a year  or  so  ago  the  Aerospace  Industry  profit  was  about  three 
and  one  half  percent.  That's  a big  number  If  you  multiply  that  by  some  billions  of  dollars* 
but  It's  not  big  on  Individual  contracts  by  any  means.  However,  application  of  A109  prin- 
cipals accotait  for  substantial  reductions  In  the  BAP  cost  which  are  handled  by  000  In  a 
manner  similar  to  the  IRAO  cost. 

P#r  the  full  benefit  Inplldt  In  A109  to  be  realized*  all  the  players*  from  Congress 
and  the  Agency*  must  sincerely  eshrace  the  concept  and  not  atteopt  to  superinpose  It  upon 
the  traditional  method  of  major  system  acquisition.  The  Navy  Ship  1 Intermediate  Combat 
system.  Certs*  Is  the  major  defense  tactical  system  to  follow  A109  procedures.  It  Is 
experiencing  some  difficulty  with  Congress*  which  seems  bent  on  supplying  a specifically 
Identified  light-weight  fire  control  system.  This  Is  not  to  suggest  that  the  one  that  Is 
being  proposed  Isn't  the  answer;  It's  Just  that  It's  probably  not  the  only  one.  To  quote 
a recent  editorial  In  the  Government  Executive  Magazine*  which  Is  produced  around  Washington 
and  always  has  a well  known  government  executive  on  the  cover*  "Industry  already  feels 
considerable  Pentagnrr pressure  on  where  to  spend  Its  IRAO  and  BAP*  the  business  proposal 
piece*  discretionary  Investment  funds."  AIQ9  Increases  that  pressure  by  bringing  then  In 
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even  eerller  for  conceptual  stuiHes.  It  goes  on  to  say  that  after  doing  that,  ^ 

In^trr  Is  told  the  Investment  1$  In  the  trash  barrel,  and  they  are  to  do  It  all  over 
again  using  the  In-house  developed  school  solution.  Industry  trill  flock  awey  fro*  A109  In 
droves. 


That  reirinds  ae  of  m Interesting  story.  This  guy  was  one  of  the  wrld's  worst  theater 


perfbnKrs.  was  a substitute  In  a Haalet  plv,  and  the  principal  In  the  pUy  111  «"<• 

~ stage  aid  the  audience  hated  hie.  He  was  such  a pompous  ass  that  they  couldn  t 


||0  Oil  StAQB  

Stand  hi*.  He  had  been  on  for  ti«ntr  or  thirty  nlnutes  and  they  began  to  throw  things 
on  the  stage,  and  he  was  annoyed  but  he  continued  on  without  Slaying  anything  and  ju^  ^t 
on  and  on  and  on  until  he  cane  to  the  famous  soliloquy.  He  was  about  halfiay  through 

' He  stopped  and 


soeebody  pasted  hie  with  a very  ripe  tomato  right  on  the  side  of  the  head.  He  stopped 
Mid  to  these  people,  "you  know,  you’re  being  very  Indecent  and  I don't  understand  idiy 
you're  doing  this.  After  all,  you  know,  I'e  only  an  actor;  I did  not  write  this  crap. 

^ ^11  .T 

If  that's  your  view  of  A109.  I don't  know,  but  I would  hasten  to  add  that  It  Is  not 
Bine.  Sowtlees,  we  are  all  sUtus  co-experts.  He  enjoy  doing  business  the  way  we  used  to 
do  business;  the  sane  old  stand,  the  same  old  people.  He  all  change  and  times  change 
drwatically.  If  you  don’t  believe  that,  just  look  at  what’s  happened  In  the  last  ten  years. 
Contract  fbrm  generally  doesn't  give  us  much  trouble  inless  we're  losing  a lot  of  money, 

■Id  you  usually  hear  about  It  idien  we  are  losing  mey.  *It’s  the  wrong  kind  of  contracti 
But  when  we're  doing  fairly  well,  we're  usually  fairly  happy.  A109.  may  perrtt  us  to 
exercise  our  vfnds  and  our  Ingenuity  to  e considerably  larger  degree  than  _*f*^*. 

a couple  of  years,  we  fall  right  flat  on  our  faces,  we’ll  have  to  coae  back  and  admit  that 
there  were  iom  considerable  problean  with  It.  I personally  believe  that  the  A109  concept 
will  prevail  once  the  Initial  roadblocks  and  problems  attendent  to  the  required  reorienta- 
tion of  thought  and  action  on  the  parts  of  all  Involved  are  overcome.  Industry,  I hope, 
stands  fully  prepared.  If  not  eager,  to  play  Ito  part  In  reducing  procurement  costs  and  w 
lead  time  and  looks  forward  to  the  rigorous  tuplementatlon  of  A109  concepto.  Thank  you 
very  *uch. 
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nrnAS:  TRAWSlATIliC  WBOiiBiBtT  iESEARCH  IffTO  AaiBr 


Harry  J.  Gray 
Chaiman,  President  and 
Chief  Executive  Officer 
Uilted  Technologies  Corporation 


!*■  going  to  continue  the  theses  Introduced  by  John  Richardson  and  Tor  Pownall. 

I rould  like  to  tell  you  about  soik  Innovative  procuresent  techniques  uhich  ererged 
fros  one  of  our  company's  major  defense  programs  — the  Utility  Tactical  Transport 
Aircraft  System,  or  UTTAS,  which  our  Sikorsky  division  Is  producing  for  the  Amy.  I 
cannot  s^gest  that  all  of  the  requirements  placed  upon  Sikorsky  are  urvfxed  blessings. 
L°?  program's  structure  favors  the  contractor  w1  th  the 

Mst  product  and  the  tightest  program  control.  Since  that  turned  out  to  be  Sikorsky.- 
I m happily  prepared  to  emphasize  the  positive. 


Pmurement  research  Is  relatively  new  — as  a structured  discipline  — In  the 
Department  of  Defense.  Many  of  UTTAS's  Innovative  program  management  and  contracting 

^ actually  u>1ng  procurement  research.  They 
Just  hadn  t been  told  It  had  a name.  Semantics  aside,  the  selection  of  the  UTTAS 
program  as  a topic  for  discussion  makes  good  sense.  The  program,  with  Its  Innovative 
corrects,  ms  developed  by  the  Any  Aviation  Systems  Comaand  In  St.  Louis,  and  It  is 


^ today  will  cover  the  program  backgrourvl  and  request  for  porposal,  the 

overall  program  schedule,  and  some  specific  features  of  the  development  and  production 

.u  detail,  but  I understand  you  are  here  to  discuss  In  depth 

IJl*  spectiT*  of  research  subjects  covered  by  the  word  'procureaent.*  You  are 
not  here  to  listen  to  broad  generalities  which  you  can't  zero  In  on. 

Actually,  the  UHAS  requirement  originated  before  the  Vietnam  war.  At  that  time 
Ar^  planners  decided  that  a utility  helicopter  — large  enough  to  carry  an  11-man 
Infantry  s^ad  — was  the  key  to  the  developing  concept  of  almobillty.  As  you 

when  fighting  escalated  over  there  the  Huey  helicopter  was 
mified  and  pressed  Into  service  as  a troop  carrier,  even  though  It  could  not  carry 
II  men  on  a hot  day.  

T'*  l«»^e-scale  helicopter  operations  In  history  took  place  In  Vietnam. 

;!!!  learned  and  concepts  tested.  As  the  shortcomings  of  existing  helicopter 

oesigns  became  apparent,  two  approaches  emerged: 

for  the  next  generation  of  helicopters  were  revised  con- 
tinually to  reflect  combat  experience.  Second,  in  areas  where  existing  technology 

*****  ?*?  projects  were  started.  Sane  research 

was  oone  at  government  Installations,  some  was  performed  under  contract  by  Industry 
and  some  was  conducted  Independently  by  Industry. 
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As  a result,  three  aanufacturers  had  the  necessary  k»0  base  to  respond  when  the 
Any  Issued  a re<}uest  for  proposal  in  1972.  This  RfP  deaanded  aore  than  had  ever 
been  achieved  In  a helicopter  — better  perfomance.  crashworthiness,  reliability, 
survivability,  and  ease  of  Maintenance.  Sikorsky  was  selected  for  one  of  the  two 
develcpaent  contracts. 

It  was  no  coincidence  that  the  technology  was  ready.  The  RFP  resulted  froa  years 
of  effective  coaaeilcatlon  between  goverracnt  and  Industry.  The  requirenents.  idille 
deaandlng,  were  realistic.  The  UTTAS  request  for  proposal.  In  fact,  was  the  nost 
specific  ever  written  for  a helicopter. 

• « } 

After  years  of  study  contracts,  research  contracts,  syi^osla,  seninars.  and 
technical  exchanges,  the  Departient  of  Defense  knew  alnost  as  such  about  nhat  the 
contractor  could  and  couldn't  do  as  the  contractor  hlaself.  Research  and  conamlcatlon 
In  pre-RFP  days  were  better  than  In  earlier  prograos.  They  clearly  contributed  to  the 
success  of  the  UTTAS  pmograa.  The  excellent  RFP  uhfch  resulted  was  a allestone  In 
practical  procurcaent  research.  ^ . 

So  in  Septeiter  of  1972.  two  aanufacturers  with  long  experience  In  the  helicopter 
business  began  developing  a Utility  Tactical  Transport.  Sikorsky  and  the  Vertol 
Division  of  Boeing  were  each  required  to  turn  over  their  prototypes  to  the  govemaent 
In  slightly  aore  than  three  years  for  govemaent  coapetltlve  testing.  This  evaluation 
was  not  only  against  each  other,  but  also  against  the  Bell  Huey  which  the  winner 
eventually  would  replace. 

The  RFP  specified  an  aircraft  tlut  could  lift  a squad  under  the  aost  unfavorable 
teaperature  and  altitude  conditions.  • .fly  It  a prescribed  distance  through  enemy 
snail  aras  fire.  . .take  extreae  evasive  naneuvers.  . .and.  If  shot  down,  protect 
the  occupants  froa  a severe  crash.  Each  coaponent  had  to  achieve  unprecedented 
reliability,  and  the  aircraft  had  to  be  aalntalned  under  austere  conditions. 

There  were  two  key  constraints  on  achieving  these  goals.  First,  air  trans- 
portability placed  a severe  constraint  on  size.  Second,  the  RFP  placed  an  even 
tougher  constraint  on  unit  airfrane  cost,  sonething  I'll  have  wore  to  say  about  later. 

In  the  RFP,  each  of  the  desired  characteristics  was  assigned  a quantitative 
target,  Measured  In  Maintenance  manhours  per  fll^t  hour,  mean  time  between  failure, 
and  so  forth.  There  were  dozens  of  characteristics  with  performance  targets.  Each 
was  separately  achievable  as  a result  of  the  research  performed  In  the  late  1960s. 

But  were  they  achievable  together,  on  the  same  aircraft,  within  strict  weight 
and  cost  limits?  In  fact,  they  proved  to  be. 

Though  the  Sikorsky  engineering  team  was  challenged  to  the  limit  of  Its  skills 
In  meeting  these  targets  — meet  them  they  did. 

The  Amy  is  to  be  complimented  on  writing  an  RFP  which  Integrated  such  a large 
number  of  performance  targets  Into  a system  which  could  realistically  be  built. 

The  Amy's  overriding  goal,  of  course,  was  an  aircraft  technically  able  to 
perform  Its  design  mission.  Almost  as  Important,  however,  were  the  needs  related 
to  cost,  survivability,  and  readiness.  Survivability,  by  the  way.  Is  a word  I 
especially  like.  Some  of  ny  experiences  as  a young  Infantry  company  cooinander  in 
Europe  during  World  War  II  gave  me  an  early  and  abiding  appreciation  of  survivability. 

1 even  like  the  sound  of  the  word. 
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th.t^T«  relate  to  un«,  consider  Just  one  of  the  requirewnts  ~ 

* 42-foot^r-second  vertical  crash  without  defoniing 
^ crushing  the  occupants.  That  is  29  miles  an  hour  straight  down)  It's 
a challengi^  design  objective  when  you  consider  the  weight^  the  trS»ini« 
engines,  and  accessories  mounted  atop  the  cabin.  (He  were  able  to  meet  this  objective). 

An  unsche<to1ed  example  of  meeting  system  goals  occurred  last  August  durino 

JltnTtJoSrSin^lt  lS!d*ro  ^ P'^totypes  was  flying  a’night^ission 

»i»4.  • forced  landing.  The  aircraft  landed  vertically 

dWrL  a dense  grove  of  pine  trees  with  trunks  up  to  six  incheTin  ’ 

50 u .Sj'V^ncJ'STLS  sJ 

sirs  "» *- » ■ 

^le  ?**  ‘•“/‘"t  program  in  which  considerations  of  life- 

cycle  costing  penetrated  all  phases  of  the  deslon.  As  a result  imnrnifrMnf § im 

«JL“:  sssfe.ss’-  •*'  '* 

to  sLe'desiJi  JmhliL*®  responded  with  ingenious  solutions 

JO  sene  oesign  prooiefiis.  There  are  many  mutual  benefits  when  a rAntrartAv  ii.>c 

r"rs;i.srcisr.j.";  :K.r5S.s  isr.„j.rr  rs, 

At  this  point  I'd  like  to  share  with  you  qy  thoughts  about  competitive  prototyping, 
out  JS  * '•"'■‘‘-fo-aht  co^HJtition  that  brought 

^ulny'fJs?**'^  technological^SaSh."  teS^flSIrationlf^J^oSSp^wa^hlJeH^  ^ 

j«  S|rp;sosi,»ofss  ;!;,im.sissk2e  ES'S.ii.isELTr;**' 

for  50  hA<fr<  ^^oe  days  later,  the  gearboxes  had  been  reconfigured,  tested 

2.  Hrz;  CL“S;,”rE;El3? •’*  ^ ■»“  ^ 
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I rfnn't  mean  to  suggest  that  all  new  systems  should  Involve  competitive  pro- 

tt>3T  “SH  Er 

however,  should  involve  a head-to-head  test  of  prototypes.  There  is 
UTTAS  represented  just  such  a technological  step. 

Now  let’s  tun.  from  the  RFP  to  tte  ov^ll  P'‘«9r*t*'52i*ias?5iS7lSng 
nmt»r  of  distinctive  features  i.'*  this  schedule.  First,  there  was  a rairiy  long 

time  between  award  of  the  prototype  contract  and  award  of  the  production 
cSaSdtTtS^velopmwtphasM  of  two  other  comietitive  prototype  programs.  unAS 
“ JSrISTtelf  iwger  tiu.  tl»  F-16  ua  roudhi,  ttic.  » 1.W  « U* 

A-10.  _ ■ • • . . ■■y  » •• 

The  extra  time  was  well  used  in  the  UTTAS  program.  As  problems  *J*.  . 

could  cope  with  them.  Accordingly,  when  the  production  contract  was  awarded  last 
December,  the  Sikorsky  entry  was  “production  ready.  /»>.  • ■ 

Another  part  of  the  schedule  has  special  interest.  This  is 
engineering  and  planning,  which  began  16  months  before  contract  award  for  both 

contractors.  . 

During  the  basic  engineering  development  phase,  it  became  apparent  that  the  20 

months  betamen  production  contract  award  ^ 

could  be  a problem.  So  the  Army  decided  to  fund  both  contractors  to  begin  pre- 
production  flanning  prior  to  the  award  of  a production  con^.  This  5®JP^“^- 

It  came  about  a year  before  the  production  proposal  was  su^i^ttod.  It  enable  us  to 
study  producibility  considerations  and  facilities  planning  in  detail  when  making 
configuration  decisions  and  in  preparing  our  production  proposal. 

Although  the  intent  was  to  protect  the  production  scheme  ^ *"** 

initiating  planning  and  procurement  actions,  there  was  another  ** 

we  had  the  time  to  develop  a number  of  innovative  production  techniques  that  will 
definitely  contribute  to  cost  control. 

Undoubtedly,  the  modest  cost  of  double  funding  this  relatively  inexpensive 
production  planning  was  money  well  invested. 

Now.  to  conclude.  I'm  going  to  discuss  some  innovative  features  of  the  development 
and  production  contracts. 

Because  of  a rigid  airframe  cost  target  of  $600,000  (in  1972  dollars),  we  were 
required  as  never  before  to  track  system  cost  as  a design  parameter  in  roal  time,  so 
we  created  a multi-discipline  team  with  the  task  of  tracking  the  cost  of  each  item 
on  the  aircraft  with  a work  breakdown  structure  number. 

This  meant  setting  cost  targets,  measuring  the  impact  of  design  and  process  changes 
on  each  component,  and  keeping  track  of  our  status  in  meeting  ^ 
target.  In  UlTAS,  the  costs  of  more  than  11,000  ^dividual  parts  had  to  be  trw^. 

The  discipline  and  procedures  this  created  were  successful  in  controlling  and  reducing 
program  cost. 

The  concept  of  specifying  a result  — or  a design-to-cost  target  — md  allowing 
the  contractor  maximun  flexibility  in  how  he  achieves  it,  enabled  us  to  dwlop  « 
effective  system  to  track,  control,  and  reduce  cost.  He  wholeheartedly  endorse  it. 

• 'But,  since  actual  costs  are  sensitive  to  changes  in  the  production  schedule. 


(• 
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allowances  should  be  wade  for  such  changes,  as  well  as  for  govenwent-dl mected 
changes  In  the  requfrenents.  specifications,  or  design. 


Grass  National  Product  Implicit  deflator  doesn't  accurately  reflect  Industry 
costs.  I am  convinced  that  price  Indices  more  closely  reflecting  aerospace  Industry 
costs  would  be  better  for  design-to-cost  escalation  adjustments.  Changes  In  the 
government  business  base  also  1at>ose  costs  outside  a contractor's  control.  So 
l^p^Hate  allowances  should  be  made.  I suggest  that  some  of  your  research  efforts 
be  directed  to  this  subject.  . , 3 

*1 

The  production  contract  also  contains  soae  Interesting  warranties  and  Incentive 
fl***^'  ^ understand  these  Items  have  been  subjects  for  procurement  research  by 
the  Department  of  Defense.  > 

* ' 

Sikorsky  has  guaranteed  hover  celling,  cruise  speed,  ml  aircraft  entity  weight.  ^ 


night  tests  will  be  performed  by  the  government  on  the  fifth  aircraft  of  the 
first  pr^tlon  year.  The  government  will  weigh  all  aircraft  produced  In  the  first 
year,  and  thw  each  year  In  the  succeeding  years.  Based-on  these  samples,  our 

^ or  decreased.  The  toUl  adjustment  Is  the  straight  sue 

®\^_'od1v1dial  adjustments.  The  adjustment,  however,  cannot  exceed  the  celling 
price  for  that  year.  ^ 


The  Ar^y  also  has  an  option  In  the  production  contract  to  purchase  a reliability 
Msumce  warranty.  * Under  this  option,  Sikorsky  would  be  required  to  perform  dmt 
1^1  repair  on  eleven  of  the  most  critical  dynamic  components  on  each  aircraft  for 
the  first  three  operating  years.  During  this  time,  Sikorsky  Is  only  obligated  to 

Sikorsky  can  Incorporate  1ii|>rovements  In  manufacture  or  repair  If 
a cost,  weight,  or  other  critical  attributes  are  unaffected 

Sikorsky  needs  no  Ar^  approval  for  Improved  manufacturing  techniques. 


« * straightforward  system:  If  the  component  falls  to  meet  the  Mean  Time 

Removal  R^ulrlng  Depot  Repair,  Sikorsky  repairs  more  components  than  planned 
and  ^kes  a financial  beating.  The  level  of  exposure  Increases  up  to  the  celling 
set  for  that  year.  Beyond  the  celling  Sikorsky  bears  the  full  cost.  If  the  com- 
ponents exceed  expected  Hean  Times  Between  Depot  Repair,  Sikorsky  earns  an  award. 


Included  under  reliability  assurance  warranty,  Sikorsky  has  to 
corrwt  deficle^les  resulting  from  defects  In  design,  workmanship,  or  materials  If 
the  ^^jclmy  Is  reported  within  a specific  time  after  aircraft  acceptance.  If  a 
defKt  Is  diswvered,  Sikorsky  must  submit  Its  recomnended  corrective  action  within 
90  days.  Such  action  Is  taken  at  no  Increase  In  the  contract  Urget  or  celling  price. 


As  far  as  I*m  concerned.  Incentives  and  warranties,  like  competition,  are  good 
business.  In  the  case  of  UTTAS,  they  were  the  result  of  a great  deal  of  bargaining, 
and  I tellm  they  will  accomplish  their  purposes.  This  is  one  element  of  program 
control  which  should  grow  over  the  next  ten  years. 
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The  UHAS  progra  fs  • ailestone  In  the  evolution  of  procuramt  research,  md 
It  will  continue  to  benefit  fra  Innovative  pro^ra  nanaseaent  by  Slkorshy  and  by 
the  AriQf.  So  far,  the  proyraa  has  been  characterized  by  professional  achleveaents  ~ 
w excellent  RFP  idilch  gave  It  a good  start,  a superior  dnlgn  tea,  and  excellent 
. nanagenent  of  govemaent  competitive  testing.  ^ * v;f,j 


As  far  as  IfeilM  Technologies  and  SIkorsIgr  go  - a pledge  the  Arw  the  best 
helicopters  that  have  ever  floa. 
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J»E  PERSON'S  TRAVELS  IN  PROCUREMENT  RF<:rAPrH 


Kenneth  0.  Griffiths 
Procurement  ibnagement  Review  Division 
CIrectonte  for  Procurement  and  Production 
US  Ar^y  Ibterlel  Develofaent  and  Readiness  Connd 
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irf  f«*  selected  coaents  (to  establish  the  perspective 

Osly  mr  wife  and  children  are  very  interested  in  my  personal  travels. 

_ . invited  to  express  a few  observations  about  (i)  »diy 

rK^ff*  "‘***^  important  in  tbe  field  or  procurenmt;  a^ 
^ceptible  to  improvcMits.  based  on  my 
personal  exposure  over  the  last  decade. 

I*ve  had  several  differat  job  descriptions  nost  of  which  have 
involved  travelinn  to  field  procurement  offices  to  conduct  inquiries 
iat^rocurement  :*elated  probleas  and  to  help  develop  iaprovements . In 
a very  «al  way,  r.t  has  all  been  a form  of  procurement  research.  Therefore, 
might  say  I as.  one  physical  example  of  the  concept  that  procurenent 
research  is  conducted  in  differing  dimensions  and  modes.  I have  had 
assignments  as  a contract  specialist;  a supervisor  of  a procurement 
organixational  unit;  and,  a graduate  student  in  the  GKU  Procurement  and 
Cdmtracting  Program  where  I did  research  for  a thesis. 

I*ve  been  a member  of  tbe  Army  Procurement  Research  Office  staff, 
the  DARCDN  Inspector  General  staff,  and  the  DA/DAROIM  Procurement 
Management  Review  staff.  Finally,  I We  been  on  many  ad  hoc  study 
teams.  ^ 


WHY  PROCUREMEKr  RESEARCH? 

Where  a real  need  for  improvement  can  be  identified,  I think  we 
^ht  to  pay  the  price  of  procurement  research.  Mhy?  Well,  the  need 
for  procurement  research  is  obvious;  but  General  Stansbury  has  just 
warned  us  that:  wo  should  not  ignore  the  obvious!  So,  let  me  state  that 
"obvious”  point  a little  more  explicitly. 

As  we  all  know,  the  vital  national  defense  mission  involves  both 
Government  and  industry;  and  the  critical  interface  between  these  two 
team  members  is  the  contract  relationship  which  in  turn  is  achieved  by 
means  of  the  procurement  function.  And— we  accomplish  Government 
procurement  by  very  standaxxiized  lavs,  regulations,  policies  and  opera- 
tional procedures,  by  hundreds  of  procurement  offices,  thousands  of 
specialists,  millions  of  individual  procurement  actions,  and  billions  of 
sheer  magnitude  gives  an  awesome  significance  to  the  work 
of  refining  and  improving  management  and  performance  of  the  function, 
even  though  most  of  it  is  accoiiq)lished  half  an  inch  at  a time. 
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•fe  need  to  intelligently  and  diligently  promote  and  support  procurement 
research  efforts  because  there  is  a very  big  potential  payoff  whenever 
we  make  a half  an  inch  of  progress.  That*s  whyl 

For  example:  one  of  my  first  research  team  efforts  resulted  in  a 
relatively  small  set  of  changes  to  the  language  of  a special  provision 
used  by  the  Electronics  Command.  Today » thousands  of  solicitations  and 
contracts  effected  by  this  organization  routinely  utilize  the  revised 
language  and  routinely  benefit  from  the  improved  contractual  relationships. 
Tliis  represents  a massive  return  on  the  investment  in  our  research. 

Similarly^  one  of  the  most  recent  of  my  personal  research  team 
involvements  substantially  affected  the  whole  procurement  program  in  the 
Republic  of  South  Korea.  lt*s  a fact:  successful  research  pays  off 
handsomely  in  the  US  Government  procurement  business.  This  logic  compels 
use  of  the  very  best  of  our  collective  talents  and  a substantial  propor- 
tion of  our  resources^ 


A large  portion  of  our  research  effort  has  been  concentrated  on 
i^>roving  the  process  of  acquiring  ouijor  weapons  systems.  Certainly  we 
have  needed  improvements  in  this  area^  and  in  tlie  last  few  years  we  have 
clearly  expanded  the  state  of  our  art. 


But  the  totality  of  the  procurement  program  involves  much  more  than 
just  the  major  weapons  systems.  As  I perceive  it,  we  need  to  improve  our 
performance  of  the  essential,  basic,  fundamental  steps  of  the  procurement 
process — the  core  activities  of  "blocking  and  tackling" — as  well  as  to 
push  the  state  of  our  art  in  major  weapons  systems  acquisition.  While, 
in  general,  we  have  performed  well  in  the  past  and  are  performing  well 
today,  we  are  far  from  being  highly  proficient  in  performance  of  our 
procurement  function.  Many  areas  are  susceptible  to  improvement. 

I have  not  defined  or  even  outlined  an  improvement  program  in  my 
mind.  At  this  state  its  just  a feeling,  a "hunch"  that  I have  that  we 
need  one.  An  I*m  certainly  not  blessed  with  having  solutions  to  the 
problem  trends  idiich  I suspect  exist.  But  1 do  know  a couple  of  things: 

First:  If  we  did  not  continue  to  diligently  maintain  our  expertise, 
our  local  procurement  organizations  and  ultimately  our  national  procure- 
ment capability  would  rapidly  deteriorate  and  we  would  soon  be  unable 
to  properly  service  our  customers.  If  you  doubt  this,  take  my  word  for 
it.  The  quality  of  a procurement  shop  can  slip  from  excellent  to  poor  in 
a matter  of  a year.  We  must  continuously  manage  our  weaknesses  and  make 
self  improvements  where  needed. 


Second 


I also  know  that  we  cannot  just  demand  a continued  high 

degree  of  proficiency  and  expect  to  get  it.  1 was  reminded  of  this 
rather  elementary  bit  of  wisdom  when  I stopped  overnight  in  Tokyo  a few 
months  ago  on  my  way  to  Seoul,  Korea.  In  the  desk  drawer  of  the  hotel 
lay  a copy  of  some  parables  of  BUDDA,  and  I glanced  at  a few,  one  of 
which  vent  about  like  this:' 


o 

o 
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o 
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o 
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->•  A man  felt  hioiself  to  be  fairly  rich  and  vranted  a house  much  the 
equal  to  that  of  his  neighbor  who  had  a three  stor>^  hcuss.  The 
man  commissioned  a carpenter  to  build  such  a house.  The 
carpenter  began  to  build  a foundation,  a first  floor  and  a 
second  floor,  preparatory  to  building  the  third  floor.  The  man 
became  angry  and  terminated  the  contract.  He  demanded  that  all 
he  wanted  was  the  third  floor,  so  that  he  could  display  equality 
with  his  neighbor.  Of  course,  the  carpenter  was  unable  to  help. 

~ as  MJDDA  said  it:  ^ • v x 

**A  foolish  man  thinks  only  of  results  and  is  impatient  with 
effort  that  is  necessary  to  get  good  results.  No  good  can  be 
attained  without  proper  effort,  just  as  there  can  be  no  third 
story  without  the  foundation  and  the  first  and  second  stories.** 


Some  areas  may  not  be  susceptible  to  improvement  by  concentrated 
research  efforts,  from  a practical  point  of  view.  But  identification  of 
specific  areas  is  certainly  one  of  the  first  places  to  start.  If  I were 
to  be  asked,  for  exa^le  by  means  of  a market  research  questionaire,  I 
would  add  at  least  the  following  entries  to  the  general  list: 

e - - 

— With  respect  to  operational  aspects: 

- we  need  to  make  source  selection  decisions  better  and  structure 
the  evaluation  factors  of  Section  D of  our  solicitations  a 
little  more  sensibly  — in  both  minor  as  well  as  major  procurement 
actions. 

- we  need  to  better  distinguish  between  term  and  completion 
forms  of  cost-plus-fixed-fee  t>*pe  contracts,  and  thus  better 
identify  what  is  to  be  bought  and  sold  via  our  CPFF  contracts. 

- of  course.  Section  E still  needs  improVeiaent.  We  still  have 
treble  stating  what  supplies  and  services  we  want  to  buy.  We 
know  what  a Work  Breakdown  Structure  is,  as  well  as  the  nature 
of  contract  line  items  and  subline  items.  But  we  are  not  univer- 
sally adept  at  merging  the  two  concepts  in  a clear  statement  of 
the  required  supplies  and  services. 


- we  need  to  better  conceptualize  our  requirements  and  correspond- 
ingly our  contract  schedules  in  procurements  of  engineering 
services,  maintenance  services,  base  operation  services,  facility 
management  services,  etc.  We  need  to  stop  using  money  as  the 
driving  factor,  even  though  that's  about  the  only  area  wherein 

we  can  be  very  specific.  (Every  time  we  do  this,  all  the  money 
that*s  avail^le  tends  to  get  spent.) 

- we  need  to  learn  how  to  contract  with  foreign  governments  much 
better  than  we  now  do. 


- KC  need  to  know  aore  about  how  to  cope  with  widespread,  pervasive 
collusive  practices  which  may  well  exist  in  offshore  procurement 
activities— i.e.,  how  to  select  sources  and  fix  prices  in  a way 
whereby  we  control  the  decisions. 

. - we  need  to  improve  our  techniques  of  contracting  for  architect- 
engineer  services. 

. we  need  to  learn  how  to  better  use  Basic  Ordering  Agreements, 
particularly  in  establishing  '•open  end*'  arrangements  with  A-E 
firms  and  obtaining  engineering  stqiport  to  production. 

- we  need  to  find  ways  to  expand  our  use  of  the  award  fee  concept, 
and  to  receive  the  benefits  to  be  derived  from  aligning  the 
cootractor*s  and  the  governments*  objectives  in  an  evolving 
pattern,  without  making  him  a personal  servant. 

- we  need  to  learn  how  to  better  utilize  exMlpatory  language  in 
both  competitive  and  noncompetitive  production  contracts  for 

' military  design  equipment  where  our  technical  data  packages  are 
/ inevitably  less  than  perfect.  Some  orginizatioos  are  skillful 
at  this  technique.  Others  are  not.  ^ ^ . ? 

— With  respect  to  performance  of  management  functions: 

• we  need  to  know  how  to  better  document  and  transfer  lessons 
learned.  For  example,  we*ve  learned  so  Mch  in  the  iMt  few 
years  about  flying  before  buying,  milestone  contracting,  managing 
change,  etc.  But  I*n  afraid  much  of  our  experience  may  have 
slipped  through  the  cracks  and  the  next  person  will  have  to 
, reinvent  the  wheel. 

- we  need  to  improve  our  issuance  of  procurement  instructions  ^ 
and  SO?*s  at  the  Chief  of  the  Purchasing  Office  level.  A few 
years  back  we  made  major  changes  and  rendered  almost  all  local 
guidance  invalid,  in  deference  to  the  ASPR,  APP,  DARCOM  PI*s 
(Army),  AFPl,  NAVPI,  etc.,  and  I think  a void  now  exists  at  many 
local  offices.  Guidance  which  ought  to  be  published  at  the 
local  level  is  not  now  entirely  adequate. 

- the  role  of  the  HPA  and  of  the  contracting  officer  still  needs 
more  clarification,  in  spite  of  the  fact  that  reams  of  material 
have  been  published  on  this  subject.  What  is  a HPA?  Arc  procure- 
ment types  really  professionals  such  as  attorneys,  engineers, 
accountants,  doctors,  economists,  etc.?  Iftiat  are  the  real 
functions  and  responsibilities  of  a contracting  officer? 

- effective  work  force  development  and  utilization  and  measurement 
of  productivity  continues  to  be  a weakness.  Right  here  at  West 
Point  the  two  key  procurement  people  are  leaving,  one  on  reassign- 
ment and  one  on  retirement.  How  can  we  assure  that  the  Superin- 
tendent will  not  now  suffer  unduly  for  want  of  a procurement 
capability?  In  the  larger  national  spectrum,  how  might  wc 
better  assure  a continued  strong  capability  to  provide  responsive 
procurement  service? 
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The  foregoing  entries  to  the  general  list  represent  only  one  Ban's 
observations.  Perhaps  it  would  be  well  to  conduct  a continual  narket 
• research  effort  to  maintain  clear  visability  of  the  viewpoints  of  our 
professionals  all  through  the  diverse  procuLreaent  program  to  know  our 
weaknesses  and  what  our  self  improvement  goals  ought  to  be.  In  this 
way,  knowledge  about  o«ir  real  goals  could  serve  to  define  our  real 
problems  which  in  turn  could  help  us  understand  our  needs  for,  aM 
perhaps  the  appropriate  dimensions  and  modes  of,  formal  research  efforts. 


Thank  you 


Oeclficallv  consider  the  dimensions  oP  urmetiynm^int’  1 V ^ ^ r ^ ^ * 
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To  aore  specifically  consider  the  diaensions  of  procureaent  research, 
let  us  now  hear  froa  Dr.  Joseph  L.  Hood. 
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SEQUCNHAL  RESEARCH  NEEDS  IN  THE 

EVOLVING  DISCI ^INE  OF  PROCURE?^* 


V, 


by 


'Joseph  L.  Hood,  Ph.D. 


^ I 


Defense  Systeas  Hanagenent  College 


Daniel  E.  Strayer,  Ph.D. 

Air  Force  Business  Research  Kanagenent  Center 


IWTRODOaiOH 

Much  of  the  effort  toward  a broader  definition  of  research  In  pro- 
curement seems  to  present  the  perspective  toward  research  In  a single 
time  dimension.  !n  It  possible  that  the  research  needs  of  a field  of 
study  and  practice,  such  as  procurement,  also  vary  over  time? 

Criticism  has  been  leveled  at  procurement  by  scholars  and  practi- 
tioners of  older  disciplines  for  the  naivete  of  that  research.  Reactions 
to  these  criticisms  have  been  a combination  of  guilt  feelings,  defensive- 
ness and  scapegoating. , It  Is  true  that  research  In  procurement  Is  mustly 
descriptive,  with  heavy  emphasis  on  surveys,  case  studies,  and  reports  of 
Individual  experience:  and  It  ought  to  be  more  analytical  and  experimental 
(guilt  feeling).  But  pioneers  who  are  so  busy  building  a new  field  don't 
have  the  time  to  engage  In  more  scientific  research  (defensiveness). 
Besides,  It  may  be  the  fault  of  the  older  disciplines  that  procurement 
research  Is  In  the  state  It  Is— they  haven’t  shown  enough  Interest  In  our 
field  to  bring  their  discipline  to  bear  on  It  (scapegoating).  The  growth 
of  a field  of  practice  like  procurement  Is  a genetic-like  process  which 
proceeds  as  If  by  natural  law  according  to  an  organically-determined 
sequence  of  phases  of  development.  If  this  is  true,  a field  of  practice 
may  have  developmental  knowledge  needs  that  change  through  the  stages  of 
maturation.  So,  what  are  the  developmental  knowledge  needs  for  procure- 
ment research? 

Institutional  leaders,  sponsoring  procurement  research,  and  consul- 
tants and  students.  In  their  constant  quest  for  research  problems  that 
would  be  relevant  to  the  needs  of  the  field  of  procurement,  sometimes 


*The  contents  of  this  paper  are  adapted  from  Halcomb  S.  Knowles* 
speculative  theory  for  "Sequential  Research  Needs  in  Evolving  Disciplines 
of  Social  Practice,"  Adult  Education,  XXIII,  4,  1973. 
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appear  to  look  Introspectively  at  problems  and  concerns.  They  seae  to 
search  the  periodical  literature  to  discover  needs  as  perceived  by  the 
leaders  of  the  field.  They  apparently  then  explore  the  research  prob- 
lems being  studied  In  related  fields  for  clues  as  to  what  the  procure-  ' 
■snt  field  ought  to  be  concerned  with.  These  are  sometimes  good  sources 
for  building  a master  list  of  possible  procurement  research  problems. 

But  what  many  have  been  asking  for  Is  a criterion  fbr  helping  them 
select  from  the  list  those  problems  that  would  be  In  te^ro  with  the 
developmental  knowledge  needs  of  the  field,  iind  this  we  have  not  had. 

I 

Toward  filling  this  void,  a theoretical  construct  of  sequential 
research  needs  In  the  evolving  discipline  of  procurement  Is  presented 
In  the  form  of  six  sequential  or  developmental  phases:  „ . 


, .1.^  Definition  of  the  Field. 

‘"■J;  B.'"  Differentiation  of  the  Field. 
^7  3.  Standard-Setting.  . 


4.  Technological  Refinement. 

5.  Respectability  and  Justification. 

f 

6.  Understanding  of  the  Pynamics  of  the  Field 
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The  following  discussion  presents  each  of  these  phases  In  terms  of 
(1)  the  kinds  of  questions  or  Issues  to  be  addressed,  (2)  the  different 
types  of  research  activity  within  the  phase,  and  (3)  exasples  of  Individ- 
ual research  within  each  phase  that  has  already  been  performed. 


‘ f 


DEVELOPHEWTAL  NEEDS  FOR  RESEARCH  IW  PBOCUREtCW* 


PHASE  1 : DEFINinON  OF  THE  FIELD 


The  first  organic  need  of  a field  of  social  practice  such  as  procure- 
ment Is  for  a definition  of  itself.  As  the  pioneers  of  a new  kind  of 
practice  start  becoming  aware  of  the  fact  that  they  are  doing  something 
different  and  start  bisnplng  Into  others  who  are  doing  much  the  some  thing, 
they  begin  to  get  a sense  of  Identification  with  one  another.  And  then 
they  start  asking  such  questions  as:  Who  else  is  doing  this  kind  of 
thing?  How  many  are  there?  Where  are  they  located?  What  types  of 
Institutions  are  they  in?  What,  exactly,  are  they  doing  and  how?  What 
are  their  objectives?  Who  are  their  clients?  What  terminology  are  they  . 
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using  to  describe  themselves  and  their  work?  Under  what  conditions  are 
they  working?  What  are  their  resources?  What  are  their  problems  and 
concerns?  What  are  their  characteristics?  In  what  directions  are  they 
moving?  'These  and  other  questions  arise  out  of  the  natural  need  for  a 
new  field  to  become  defined.  The  field  of  procurement  practice  needs 
to  be  described  before  It  can  be  presented  with  Integrity* 

Accordingly,  during  this  phase  the  great  need  Is  fbr  descriptive 
research— descriptive  surveys,  census  studies,  case  reports,  daaographic 
studies,  and  the  like.  These  studies  aim  to  establish  the  boundaries  of 
the  problem,  sharply  delineate  the  character  and  extent  of  Important 
aspens  of  the  situation  or  context,  and  establish  a baseline  for  fur-* 
ther  analysis  and  managerial  action. 

The  studies  are  frequently  done  by  staffs  as  part  of  larger  projects, 
e.g.,  the  initial  survey  of  the  Air  Force  pricing  capability  done  by  the  • 
Air  Staff  and  major  commands  under  the  COPPER  IWACT  Project  (1),  and 
research  studies— Hitt's  study  describing  the  priorities  and  functions 
performed  by  the  Base  Procurement  and  Contracting  Officers  (2)  or  Goss 
and  Lockwood's  description  of  the  priorities  and  functional  respon- 
sibilities of  SPO  versus  AFPRO  production  functions  (3).  In  other  cases, 
descriptive  research  Involving  regulation  and  library  research  has  been 
necessary  to  begin  deeper  study  of  a par^cular  area  of  management  con- 
cern. In  the  area  of  Foreign  Hilttary  Business,  studies  by  Hatema  (4), 
and  by  Carver  and  Walsworth  (5),  attested  to  describe  fully  the  pro- 
cedures, responsibilities,  and  major  actors  Involved  In  the  DD  Form  1513 
processing  sequence  and  the  NATO  supply  system. 

While  nore  of  these  studies  could  be  considered  conclusive,  most 
did  not  use  hypothesis  testing,  and  none  employed  the  utmost  In  meth- 
odological rigor,  they  have  ea^  provided  Important  knowledge.  They 
have  provided  a baseline.  Additional  research  can  be  structured  around 
the  gaps  and  problem  areas  established  by  the  Initial  studies.  Initial 
definitions  of  the  terminology  and  functions  characteristic  of  the  field 
studied  are  offered  for  the  use  of  future  researchers  and  the  next 
researcher  has  a beginning  conception  of  the  area  of  study  which  enables 
him  to  proceed  at  a more  rapid  rate  than  would  be  possible  otherwise. 

As  additional  research  Is  concentrated  Into  an  area  of  concern,  the 
research  becomes  more  specialized.  Terms  become  standardized,  and  both 
researchers  and  practitioners  become  aware  that  they  are  working  In  a 
distinct  field  which  they  can  describe  and  analyze  more  rigorously. 

When  this  process  has  matured,  the  field  of  study  may  be  said  to  be 
differentiated.  It  now  enters  Phase  2. 
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PHASE  2:  OIFPEIOTIATION  OF  THE  FIELD 


As  « field  of  practice  becomes  identified,  it  begins  to  be  differen- 
tiated from  other  fields  of  practice  and  relationships  with  them  (such 
as  procureoKnt  with  the  fields  of  accounting  and  law)  beccae  clarified. 
Questions  such  as:  Exactly  how  is  it  different  from  other  related  fields 
of  practice  in  its  goals,  values,  auspices,  clientele,  and  methodology? 

In  what  ways  does  it  coi^iete  with  them  versus  complement  them?  Miat 
unique  needs  is  it  meeting  that  can't  be  mei:  equally  well  by  established 
fields?  Uhat  right  does  it  have  to  claim  special  resources  for  itself? 
Nhat  specialized  training  or  talents  are  required  to  engage  in  its  field 
of  social  practice?  must  be  answered.  . ; ’ 

During  this  phase  there  is  need  for  (1)  comparative  studies  that 
delineate  roles  and  technologies  among  the  fields  of  practice.  (2) 
exploratory  research  that  probes  boundaries.  (3)  reports  of  artistic 
practice  that  establish  uniqueness  of  approach,  and  (4)  analyses  of  need. 

Frequently,  at  this  stage,  the  function  concerned  with  performance 
has  become  differentiated;  and  Job  descriptions,  etc.  are  now  more  ti^tly 
drawn.  Parallel  staff  elements  have  frequently  been  formed  and  the  ques- 
tion of  training  becomes  of  concern.  . . 

Perhaps  the  clearest  example  of  this  type  of  an  area  is  that  of  con- 
■ tract  pricing.  In  the  Air  Force,  pricing  has  become  a differentiated, 
highly  skilled  subset  of  the  procurement  function.  Price  analysts  are 
recogniz^  as  requiring  particular  talents  and  analytic  skills.  Special 
courses  have  been  set  up  under  the  COPPER  IJPACT  Program  to  improve  and 
maintain  these  skills  and  the  respective  staff  elements  and  schools 
maintain  a healthy  interest  in  these  matters.  Knowledge  about  the  pric- 
ing activity,  however,  remains  inmature.  There  is  need  for  additional 
studies  to  place  a firmer  foundation  under  important  aspects  of  the 
function. 

There  is  currently  axjch  emphasis  on  research  involving  models. 
Frequently  the  models  are  differentiated  with  regard  to  specific  areas 
of  concern,  such  as  the  model  developed  by  Ueida  (6)  for  predicting 
R4D  costs,  or  that  of  Karsch  (7)  for  estimating  total  system  cost. 

New  models  are  introduced  to  direct  attention  to  variables  that  have 
been  identified  by  previous  models  but  not  satisfactorily  treated. 

The  development  of  the  Entropic  Cost  Model  by  Martin  (8)  and  the  sub- 
sequent case  study  tests  of  the  model  Glover  and  Lenz  (9).  and 
Babiarz  and  Giedras  (10).  exemplifies  this  process. 

As  the  field  becomes  better  understood,  subpractices  begin  to  develop; 
and  specialization  in  these  is  observed.  As  research  demonstrates  its 
ability  to  improve  performance,  functional  staffs  become  concerned  with 
the  problem  of  maintaining  currency  and  improving  the  credibility  of  field 
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perforaance.  At  some  point  in  this  process,  the  field  frequently  enters 
Phase  3,  Standard-Setting. 

v'y.:  V -J  '•  ' : i-5  ' JVr'r  f,  r.ii 

f.?'  Jr.‘  - • •-  ••  • 

tj  ; . PHASE  3:  STANDARD-SETTING  . ..  , ^ 

' Once  a field  of  social  practice  is  clearly  defined  and  differenti- 
ated fron  other  fields  (in  a sense,  once  its  right  to  nenbership  in  the 
applied  social  sciences  is  established),  the  problen  of  control  arises. . 
Questions  such  as:  Uhat  are  the  standards  of  practice  now  observed? 

Uhat  should  be  the  mniinui  standards  of  practice?  Hhat  outcones  are 
actually  being  achieved  through  its  practice?  Hhat  are  the  appropriate  ' 
criteria  for  evaluating  the  effectiveness  of  its  practice?  Uiat  pro- 
cedures should  be  used  for  neasuring  its  effectiveness?  Hhat  sanctions 
are  available  and  effective  in  niintaining  accepted  standards?  How 
should  training  institutions  be  accredited  and  practitioners  be  certi- 
fied? Mist  be  addressed. 

During  this  phase  the  greatest  need  is  for  (1)  noraative-descriptive 
research  which  yields  insight  as  to  desirable  standards,  (2)  evaluative 
research  which  appraises  the  ootcooe  of  both  training  and  practice,  and 
(3)  instniaental  research  tdiich  provides  inproved  tools  and  procedures 
of  measui^enent. 

Research  is  frequently  shifted  back  to  staff  organizations  for  per- 
formance at  this  stage.  As  recognition  and  acceptance  of  the  area  grows, 
the  means  of  institutionalization  are  further  explored. 

Frequently,  one  of  the  primary  means  of  institutional  Identification 
is. begun  or  at  least  seriously  considered;  establishment  of  a certifica- 
tion program.  Thus,  we  find  certified  public  accountants,  certified 
professional  engineers,  certified  teachers,  and  certified  professional 
contract  managers.  All  of  these  involve  programs  which  establish  the 
standards  to  be  exemplified  by  practitioners.  Testing  procedures  are 
involved,  and  depending  on-  tfv-  degree  of  recognition,  are  sometimes 
operated  by  governmental  bodies  such  as  those  used  by  accounting,  law, 
medicine,  etc.  Educational  curricula  assune  greater  importance,  and 
the  certification  bodies  and  other  organizations  are  entrusted  with 
insuring  that  education  remains  both  current  and  pertinent  to  the  needs 
of  the  profession. 

The  DMET  System  within  OOD,  the  emerging  concern  with  curricula  on 
the  part  of  the  Federal  Procurement  Institute;  and,  of  course,  the  recent 
professional  certification  (Certified  Professional  Contract  Manager) 
exemplify  this  feature.  As  the  staffs  and  professional  cadre  perceive 
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their  limitations  more  clearly,  they  initiate  movement  into  the  next 
phase.  Technological  Refinement. 


PHASE  4:  TECHNOLOGICAL  REFINEMENT 


* - - - • . ^ 4 ^ ' V. 

As  a field  begins  to  get  feedback  from  evaluative  research,  areas  of 


weakness  are  discovered  in  its  technology.  Many  of  the  methods  being 
used  are  found  not  to  be  producing  the  desired  outcomes.  And  so  a need 
develops  for  improvement  of  technology  (such  as  the  use  of  new  procure- 
ment techniques). 


OuHr.g  this  phase,  the  need  is  for  (1)  experimental  research  which 
tests  the  relative  effectiveness  of  different  approaches,  (2)  case  studies 
wich  deepen  the  understanding  of  the  dynamics  of  the  technology,  and  ’ 

(3)  action  research  which  continuously  infuses  the  technology  with  the 
Insfghu  of  reality.  In  this  phase,  the  models  of  Phase  2 become  much 
more  rigorously  tested  and  more  appeal  is  made  to  empirical  analysis. 

Mto  bases  are  refined  and  become  sufficiently  well  agreed  upon  to  pro- 
vide the  basis  for  more  powerful  research  methodology.  Hypothesis  test- 
ing research  begins  to  appear  and  is  directed  at  establishing  the  efficacy 
of  the  nodels  and  defining  clearly  the  criteria  for  their  use. 

Ifithin  the  area  of  pricing,  for  example,  research  now  is  aiming  at  a 
more  complete  understanding  of  the  effects  of  different  means  of  treating 
inflation  (Gaver  and  Geisel  (11),  Brush  (12),  et.  al.).  Contract  incen- 
tives have  been  much  studied  (Belden  (13),  Parker  (14),  Hunt  (15),  et. 
al.),  and  research  has  begun  to  concentrate  on  the  ways  to  obtain  best 
results  from  award  fees,  perfonnance  incentives,  cost  incentives,  etc. 
(Runkle  and  Schmidt  (16),  et.  al.).  As  the  practitioners  and  managers 
involved  with  the  area  of  knowledge  gain  confidence,  they  begin  to  per- 
ceive the  need  for  a different  type  of  research.  Knowles  identifies 
these  new  studies  as  legitimation  efforts  aiming  at  Phase  5,  Respectabil- 
ity and  Justification. 


PHASE  5:  RESPECTABILITY  AND  JUSTIFICATION 


As  a field  gains  the  stability  that  comes  from  definition,  differen- 
tiation, standard-setting,  and  technological  refinement,  a need  for 
status  and  esteem  develops.  Needs  for  survival  and  safety  have  been 
satisfied  and  now  the  emphasis  is  on  recognition  as  a field  worthy  of 
respect. 

^ring  this  phase  the  need  is  for:  (1)  historical  research  which 
provides  the  respectability  of  accumulated  expedience,  (2)  biographical 
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stuales  which  cast  the  aura  of  Illustrious  figures  on  the  field*  and 
(3)  field-evaluative  studies  which  demonstrate  the  effectiveness  of 
the  field  In  accomplishing  Its  goals.  During  this  phase*  also*  there 
IS  a need  for  a more  sophisticated  round  of  survey-descriptive  and 
comparative  studies  to  shew  how  far  the  field  has  matured  since  Its 
original  definition  and  differentiation. 

Many  exa»g)les  can  be  cited  from  within  the  procurement  practice  to 
exespllfy  this  phase.  The  Conalsslon  on  Govemnent  Procurement  was  an 
example.  Beginning  with  a rigorous  view  of  the  procurement  practice  as 
It  existed*  the  commission  established  a library*  perfonmid  extensive 
research*  and  made  numerous  recoamndatlons.  NaiQf  conalsslon  staff  mem- 
bers have  gone  on  to  achieve  Illustrious  reputations  In  the  procurement 
field.  Conferences  and  qrnposla  become  established  featuring  research 
Into  the  field  and  the  provision  of  research-oriented  findings  to  the 
attendees*  as  opposed  to  policy  promulgation  or  ‘how  to‘  subject  natter. 
The  founding  of  the  DOD  Procurement  Research  Symposlua  In  1972  and  Its 
continuation  exe^>11fy  tols  feature  of  Phase  5.  With  continuing  research 
shedding  light  on  probleas  and  providing  tantalizing  hints  of  hm  better 
methodologies  might  be  eag>1oyed  to  cope  with  these  probleas,  the  field 
becoems  more  aware  of  Its  Internal  structure  and  also  of  the  external 
forces  that  shaoe  It.  It  thus  moves  Into  the  last  and  continuing  phase 
of  a mature  field;  the  sixth  phase*  Understanding  of  Its  Own  Dynamics. 


PHASE  6:  UH0ERSTANDIN6  OF  THE  DYKAMICS  OF  THE  Fiao 

• < -i 

Once  a field  has  become  well  established  and  Is  esteemed*  the  need 
Is  perceived  to  understand  the  Internal  and  external  forces  that  are 
affecting  Its  development.  It  now  raises  such  questions  about  Itself 
as:  What  are  the  functional  elements  of  the  field  and  how  should  they 
be  organized  Into  a unified  system?  What  are  the  resistances  to  change 
In  the  field?  What  are  the  changes  In  society  to  which  the  field  should 
be  responding?  Uhat  are  the  societal  models  the  field  should  be  trying 
to  work  toward?  What  are  the  processes  by  which  the  direction  of  move- 
ment of  the  field  Is  determined;  and  what  should  they  be? 

During  this  phase*  the  need  Is  for:  (1)  Institutional  studies  which 
will  shed  light  on  the  Internal  structure  and  stresses  of  the  field*  (2) 
environmental  studies  which  will  Identify  societal  trends  to  which  the 
field  should  be  respondlno*  (3)  force  field  analyses  which  will  reveal 
resistances  to  change*  (4)  systems  analyses  which  will  Indicate  the 
Interrelationships  among  the  elements  of  the  total  system*  and  (5)  pre- 
diction studies  which  will  project  alternative  directions  of  future 
movement  and  test  their  consequences. 

The  evidence  suggests  that  the  field  of  procurement  practice  Is*  In 
some  aspects*  entering  Phase  6.  The  emergence  of  the  concept  of 
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acquisition  procurenent,  eirbodled  In  the  Comnlsslon  on  Goverranent  Pro- 
curement's report  and  In  OHB  Circular  A-109  Is  an  Indication.  Studies 
such  as  those  performed  by  Wa1<taan  and  Rutledge  (17)  and  Pratt  and 
Marshall  (18)  on  the  Identification  of  laportant  variables  beaiir^  on 
the  negotiation  process  and  of  the  Identification  of  the  llnkages^of 
these  variables  to  organizational  elements  become  appropriate  to 
Identify  Important  Interrelationships.  The  existence  of  these  and 
promotion  of  further  studies  Into  organizational  Interrelationships 
mqy  be  regarded  as  exemplifying  the  field's  arrival  and  maturation. 

Thus,  based  on  Knowles'  conception,  the  field  of  procurement  has. 

In  some  respects,  reached  maturity.  This  happy  state  is  not.  however, 
reached  across  the  board,  nor  Is  It  a continuing  state  that  once 
reached  can  be  expected  to  persist  indefinitely.  Indeed  the  opfiosite 
is  true.  Because  the  field  is  recognized  as  a profession,  greater 
responsibilities  are  placed  on  it.  . 

As  its  understanding  of  its  relationships  with  other  functions 
grows,  the  profession  is  faced  with  the  need  to  Increase  its  knowledge 
and  to  maintain  pace  with  the  Important  outside  forces  frequently 
generated  by  collateral  functional  areas.  As  knowledge  advances, 
subareas  within  the  profession  are  identified,  and  begin  the  quest  for  ' 
complete  understanding  of  themselves.  Thus,  at  a given  time,  research 
within  the  practice  can  and  should  be  found  exeapllfylng  each  of  the 
preceding  phases.  In  fact,  the  character,  quality,  and  extent  of 
research  under  way  nay  be  regarded  as  an  Important  Indicator  of  the 
practice's  vitality. 

This  conception  of  the  developmental  needs  for  research  in  a field 
of  practice  such  as  procurement  can  be  suasarlzed  schematically  as 
follows: 


Phase 

Organic  Need 

Relevant  Research 

1 

Definition  of  the  Field 

Survey-descriptive  studies 
Census  studies 

Case  reports 

Demographic  studies 

2 

Differentiation  of  the  Field 

Comparative  studies 

Exploratory  studies 

Reix)rts  of  artistic  experience 
Need  analysis 

3 

Standard-Setting 

Nonaative-descriptive  studies 
Evaluative  research 
Instrumental  studies 

f A V- 


Phase 


Organic  Need 


Relevant  Research 


Technological  Refinenent 


Experimental  research 
_ _ ...  Case  studies 

.;  ..•li:fTn<r»'.<*45  Jittai-Theory-buildlhg'  iir  -.  . 

.-  ••,.>»  5t-5i»  ?3  fiivi  .igntftotjf Action-research  . — ;r--  - 

ar  ■ 4;ar.  j ^ ’ -‘..t ! ^ o f *«•*■  rj < 

5 Respectability  and  Justification  Historical  studies  : 

Biographical  research 

^9  ^>ao'T  cAsUtt^o)  Field-evaluative  studies 
*»ivds.*vt  7*  Sitsir  if((T  SurvejMlesciiptlve  studies 

• r'yv:  T^'tr  »tiJi  s;viv-!rt?nco  s ji  ir  ft  Comparative  studies  j" 

yfii  r>j  c*i » j'.V.-'’  vt 

6 * Understanding  of  the  Dynamics  of  ' > Institutional  studies 

the  Field  >vt  av  Envlrommntal  studies 

Force-field  analysis 

*rKlt-o’d?^  ti(is(sftcli^{9x  iit  Systev  analysis  ' 

31^  53i»«g1.o?  Prediction  studies  ■ 
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There  are  two  qualifications  to  the  presentation  of  this  construct 
so  far.  First,  the  phases  should  not  be  viewed  to  be  as  distinct  and 
separate  as  this  classification  oakes  them  appear.  It  may  be  necessary 
to  have  overlappli^  among  two  or  three  adjacent  phases.  For  example, 
while  a field  Is  working  on  Its  definition,  there  nay  be  some  work 
going  on  legitimately  on  differentiation  and  perhaps  even  on  standard 
setting. 

In  the  second  place,  the  phases  should  not  be  considered  as  being 
linear;  rather,  they  should  be  viewed  as  being  spiral.  An  evolving 
field  nay  need  to  move  through  the  six  phases  a< first  tine  fairly  super- 
ficially and  then  to  repeat  then  In  ever  deeper  cycles.  The  tine  span 
for  eadh  cycle  nay  become  shorter  and  shorter. 


USES  OF  THIS  CONSTRUa 


This  theoretical  construct  Is  presented  In  the  hope  that  It  will 
be  used  In  at  least  two  ways. 

First,  It  should  be  put  to  a rigorous  intellectual  test.  How  do 
these  speculations  stand  up  In  the  light  of  the  experience.  Intuition, 


I 


and  logic  oossessed  by  the  procurement  comnunity?  , Does  the  iiotion  of  | 
genetically-deteiwined  phases  of  development  of  a field  of  practice  I 
Mke  sense?  If  not,  what  alternate  guidelines  to  the  suggested  pat-  1 
teming  of  research  make  more  sense?  If  the  idea  of  the  developmental 
phases  makes  sense,  how  do  the  six  phases  hold  up  to  analysis?  Is  the , 
sequence  correct?  Should  other  phases  be  added?  Have  the  relevant  » 
types  of  research  for  each  phase  been  identified? 


.1 


Second,  the  construct  should  be  tested  empirically.  Institutional 
leaders,  as  well  as  researchers,  should  try  to  apply  It  to  the  selection 
of  procurement  researdi  problems.  Moreover,  they  should  report  whether 
or  not  It  holds  up  as  a criterion  of  relevance  for  pit>cureaent  research 
la  Its  different  developmental  and  environaental  stages. 

•4  I , • ••  " ■ ' • ‘ . .V-.u 

IMPLlCATlOlg  4 ; . ^ . 

As  a way  of  organizing  our  knowledge,  the  Knowles*  construct  has  ^ 
hqnrtant  i^>11cat1ons  for  procurement  Institutional  leaders.  Public 
demand  for  Improved  efficiency*  effectiveness,  and  accuracy  In  procure- 
ment Is  frequently  expressed  as  a need  for  Improved  procurement  policy  - 
and  procedui^.  Research  studies  providing  better  understanding  and 
analysis  of  the  procurement  process  can  contribute  to  meeting  the 
public's  expectations.  Management  can  act  or  refuse  to  act.  using 
research  findings  .s  an  important  part  of  the  decision  process.  In 
fact,  some  of  the  examples  cited  for  each  of  the  developmental  phases 
;ave  Influenced  changes  In  procurmnent  policy,  practice,  and  organiza- 
tion. 

However,  what  can  be  accomplished  through  research  Is  different 
depending  on  the  knowledge  base.  Sophisticated  n;ode1s  cannot  be 
developed  until  a rigorous  and  agreed  on  description  of  the  pertinent 
aspects  of  the  system  is  available:  accepted  standards  will  not  be 
forthcoming  until  normative  research  has  become  an  integral  part  of 
the  knowledge  base,  etc.  Thus,  the  research  of  the  past  provides  the 
framework  by  which  we  understand  the  present  situation  and  identify 
the  future's  needs.  By  focusing  our  research  activities  on  the  most 
pressing  knowledge  needs  and  avoiding  premature  commitments,  management 
can  set’ the  stage  for  orderly  improvement. 
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DETERMIHINS  IWAT  TO  PROCURE 


United  States  General  Accounting  Office 
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GAO’S  role  In  "procurement  research*  Is  an  Indirect  one  hy  virtue  of  Its  unique 
position  In  Governaient.  That  demands  some  explanation  and  even  In  this  audience  some 
background  on  GAO  Is  In  order. 


■"  r 
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The  purpose  of  the  Budget  and  Accounting  Act  of  1921  was  to  provide  a national 
budget  system  and  an  Independent  audit  of  Government  accounts.  To  accomplish  these 
goals  the  act  established  a new  Bureau  of  the  Budget  to  compile  the  President’s  budget, 
and  created  the  United  States  GenereT  Accountlag  Office,  Independent  of  the  executive 
departments.  GAO  Is  headed  by  the  Comptroller  General  of  the  United  States  who  Is 
appointed  by  the  President,  with  the  advice  and  consent  of  the  Senate,  to  serve  15  years. 
He  nay  be  removed  during  his  term  only  by  Impeachment  for  Incapacitation.  Inefficiency, 
neglect  of  duty,  malfeasance  In  office,  commission  of  a felony,  or  conduct  Involving 
moral  turpitude.  Therein  lies  the  basis  for  the  non-political  objectivity  of  the  GAO. 


SLIDE  3 


Authority  Is  broad 


SLIDE  4 


From  Its  Inception  the  GAO  has  been  responsible  for  determining  the  legality  and 
accuracy  of  receipt,  disbursement,  and  application  of  public  funds  SLIDE  5 with  the 
objectives  of  greater  economy  and  efficiency  in  public  expenditures. 
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Subsequent  practice  and  legislation,  confirmed  In  the  Legislative  Reorganization 
Act  of  1970,  made  the  GAO  responsible  for  reviewing  the  performance  obtained  from  public 
expenditures  - the  objective  in  this  case  Is  increasing  the  effectiveness  of  public 
expenditures. 
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The  responsibilities  are  carried  out  In  many  ways.  The  more  significant  are: 
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1.  Financial  and  Conpllance  ^Its.  ’.■  ;^L  - 

2.  Hanaganent  Re¥la»s  of  Econonur -and  Efficiency. 

3.  Progran  Results  Reviews.  - 

■ ' I k'  ■ •■?  • “* 'itJJ  i .s'.;,  yj'.*.:'*  -j  ; I’'  r^j.i^J] 
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My  of  you  Mtio  have  been  subject  to  GAO  audit  know  what  these  categories  entail. 


Other  vehicles  include: 


' Published  and  Unpublished  Decisions  of  the  CiMptroller  General  on  new  or  novel  ■ 
questions  of  law,  or  out  of  the  ordinary  circumstances. 
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Govemaent  Contract  Bid  Protest  Decisions. 

Participation  In  the  Joint  Financial  Nanagenent  Inprovenent. 
Prograu. 

Participation  In  the  Cost  Accounting  Standards  Board. 
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Me  help  the  Congress  In  Its  legislative  oversight  function  by  providing  Its 
comittees  and  menbers  with  tliiely  Infornatlon,  analyses,  and  recomoendatlons  concerning 
the  operations  of  Govemnent. 

Nr.  Staats  feels  our  greatest  contribution  Is  to  provide  answers  to  such  questions 
as: 

—Is  It  possible  to  ellninate  waste  and  Inefficient  use  of  public  aonles? 

—Are  Federal  prograns,  whether  administered  directly  by  the  Federal  Government, 
or  through  other  organizations  such  as  State  and  local  governments,  achieving 
their  objectives? 

—Are  there  other  ways  of  accomplishing  the  objectives  of  these  programs  at  lower 
costs? 

—Are  funds  being  spent  legally  and  is  the  accounting  for  them  adequate? 

He  h^s  suniTjed  up  our  mission  as  "recommending  ways  to  make  Government  work  better"— doing 
this  as  much  as  possible  by  making  the  results  of  our  studies  known  before  options  are 
lost. 


\ 
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1 P**  firther  along  a major  acgulsltlon^  for  example,  proceeds  the  Greater  the 
relucUnce  to  reconsider,  revise,  defer,  terminate  ^ 

SLIDE  10  . * “ . 

mny  Jry52"LirhrfIIlfi?f?  ’**'*•  pwided  ^.eiicy  coverage. 

DeveloM^  Defense  Dlvlsfon  trtitcli  exanined  000*$  RKMixh  and’*^ 

?^*!?1t1oni  ^•*”^*^***  thnagewnt,  Procurewnt.  and.  beginning  In  1969,  Ihjor 
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rr.nliH^t^TT”^  ««Kl  SysteK  Acquisition  Division  was  established  as  part  of  a major 
realignmmt  In  the  organizational  structure  of  GAO.  effective  April  3 19W  Tlw  r^ 

iXmZ  3ivif?oS1i;^  study  •whicTJ2^iK’tit’?2*.cS««;;g 

to^acSler^^thl  ?*  Wfied  on  a function  and  progran  basis.  In  part 

10  accelerate  the  growth  of  functional  and  program  expertise  among  senior  sUff . 
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Back  In  1967, 
this  continues. 

SLIDE  OFF 
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»br.  staats  began  a program  to  get  greater  diversity  In  the  staff  and 


II  ****  conceived  as  one  the  functional  divisions,  with  rKnnn«fMii»f^  srannlnn 

a%  »l,the  ProcurenSSTfunctlon.  and  th^  science  tShnolSy  K^^c?^ 

bS  don^^??J  guloance.  and  comminlcatlon  on  what  has 

#*"««"*•  ;*  "*'"9  done,  and  what  should  be  done  relative  to  these  twn  issu. 

(of  29  currently)  and  serves  as  advisor  to  all  otheTeloncnU^  W. 

*1^"  ^ i<tent1f1cat1on  of  needs  and  establlstaent  of 

requirements,  (2)  the  identification  of  technology  gaps,  (3)  the  suDoort  of  re^Mrrh  ahh 

jSt  requisite  quality,  within  the  time  needed,  at  the  most  ad^^^ 

cost,  through  the  use  of  contracts,  leases,  and  grants.  «igewjs 

We  do  this  through: 

(1)  Government-wide  and  multiagency  functional  audits  of  — 

.1  Overall  policies,  practices,  and  programs  of  the  Federal  Goveriwnf 

relating  to  procurement  Including  the  acquisition  of^  Ite^  Government 

(b)  Overall  policies,  practices  and  programs  of  the  Federal  Govem^nt 
research  and  development  Including  the  Federal  Govemnent's  role  for  aidino 
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(2)  Nanageaent  and  program  audits  of  the  — 

(a)  National  Aeronautics  and  Space  Adalnlstratlon. 

(b)  Weapons  procurement  activities  of  the  Energjr  Research  and  Development 

Agency. 

(c)  Procurement  functions  (exclusive  of  the  Public  Buildings  Service 
construction)  of  the  General  Services  Atelnistratlon. 

(d)  Renegotiation  Board. 

(e)  Functions,  programs,  and  activities  of  the  Department  of  Defense,  not 
specifically  assigned  to  other  GAO  divisions. 


(f)  Other  activities,  boards. 
Government: 


■Isslons.  and  ci 
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Arsed  Services  Procuresent  Regulation  (ASPR)  Revisions. 

Central  000  overhead  agencies  (e.g.  OSO.  OLA,  NSA). 

Ccsilttee  on  Purchases  of  Blind-Hade  Products. 

Inplesentatlon  of  Rccoonendatlons  of  the  Connlsslon  on  Govemnent 
Procurenent. 

Emergency  Loan  Guarantee  Board.  \ 

These  agencies  are  responsible  for  nost  of  the  Federal  Government's  annual  expendi- 
ture for  procurement  and  science  and  technology.  In  addition  to  the  direct  activities  of 
the  Federal  agencies*  grantees*  using  Federal  funds*  procure  a large  but  unknown  amount 
of  goods  and  services  and  contribute  significantly  to  the  advancement  of  science  and 
technology. 

Another  of  our  responsibilities  Is  monitoring  compliance  with  0MB  Circular  A-76! 
SLIDE  13 

In  FY  76  we  spent  about  lOX  of  our  resources  on  procurement  as  an  Issue  area  but 
that  figure  Is  misleading.  The  other  divisions  do  work  that  includes  procurenent  issue 
coverage  but  they  Identify  the  work  by  their  own  issue  area. 

SLIDE  OFF 

There's  a lot  of  lee-way  In  what's  appropriate  to  do  at  any  point  In  time.  We 
need  to  be  sensitive  to  changing  times  and  put  out  resources  to  work  where  we  see  the 
chance  for  making  the  biggest  contribution.  Today*  certain  recurring  themes  appear  on 
the  national  scene  which  should  trigger  our  planning.  For  example; 


—The  low  level  of  public  confidence  In  the  Integrity  of  Government,  and  particularly 
in  contracting  activities,  would  seem  to  dictate  our  addressing  this  Issue- 
bearing  the  air  where  appropriate,  exposing  looseness  In  controls  and  abuses 
where  found,  and  supporting  reasonable  safeguards,  where  needed. 

^fenw  costs  during  peacetime  would  suggest  opportunities  for  our  ’ 
participatlw  with  Congress  In  a closer  scrutiny  of  the  Administration's  Defense 
strategy  and  spending  rationale— particularly  Involving  costly  and  sometimes 
questionable  or  oversold  weapons  systems. 

(r  i. 

happws  to  be  the  Issue  of  the  ■oaent,  how  iwch  we’re  working 
wi^o  the  of  public  thought,  or  how  well  we’re  anticipating  congressional 

needs,  we  still  wight  not  be  effective.  To  borrow  a thought  fro.  Mr.  Staats: 

*«.  rather  Issue  concise  and  Infonutlve  letters,  getting  thea  Into  appropriate 

***  **  point  to  stacks  of  bound  reports—all  too  late 

our  best  product  Is  a tieiely  report  to  the  Congress  containing. 
recoMendatlons  for  1^>roveMnts  to  procureaent  pol  Ides  and/or  practices.  • 

BAiirf!L^l«!!^»  trying  td  contribute  to  1^)rovements  In  Procurement 

policies  and  practices.  We  do  this  basically  in  two  ways: 

— By  provl^  Information  and  Independent  evaluations  to  Congress,  Its  coailttees 
and  aembers  m high  visibility  procurement,  and  science  and  technology— Includlno 
major  acqulsltlon-prograas^and  Issues,  and 

—By  continually  chipping  away  at  significant  management  weaknesses. 

>*■  "ot  sure  I could,  or  tWNjId  want  to  attempt  to  list  our  contributions  to 
prMure^t.  Pe^ps  you  are  the  only  ones  who  can  do  that.  I find  It  Interesting  that 
only  about  40  GAO  reports  are  Included  In  the  procurement  research  guide. 

SI.IDE  14  . -V  V - 

‘ « **  on  and  that  is  that  we  contribute  to  more 

effective,  efficient  and  economical  Government  because  GAO  exists  and  Its  presence  Is 
recognized. 

- dPcunents  In  the  lobby  for  those  who  are  interested,  and 

provldra  Bob  Williams  with  a listing  of  some  300  reports  we  have  Issued  from  July  1974 
to  April  1977. 

• V ^ • 

SLIDE  15  - . . 


— . •■**<"9  cost  of  major  weapon  systems,  which  continues  to  rise.  Inflation 

not  withstanding,  led  to  the  creation  of  the  major  acquisitions  subdivision. 


SLIDE  16  \ ^ , 

* • ■ • . ‘ ’ 
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The  initial  efforts  reported  only  on  the  cost,  schedule,  and  perfomnce  of 
selected  individual  systeas.  We  soon  broadened  our  reviews  to  include  functions  which 
cut  across  all  systems  - cost  estimating,  program  management,  cost/benefit  studies,  test 
i evaluation,  and  program  management. 


And  in  1973  we  began  reviewing  major  civil  acquisitions. 


.♦v  i - 


SLIDE  18 


Here  are  the  dos  and  don'ts. 


The  Omqitroller  General  and  the  Commission  have  both  sUted  that  the  biggest  proble 
is  determining  what  to  procure  and  for  what  inirpose. 

The  Camission  noted  that  problems  encountered  during  procurement  are  rooted  in 
action  or  inaction  of  earlier  phases*”the  identification  of  needs,  establishment  of  re* 
quirements,  or  “front-end"  of  tiie  acquisition  process.  The  latest  GAO  report  mentioned 
in  the  procurement  research  guide  is  our  1974  report  on  Identification  of  Needs  and 
Establishment  of  Requirements  for  Major  Weapon  Systems. 

SLIDE  19 

I like  to  consider  the  familiar  OOD  pipeline  chart  as  representing  the  major 
acquisition  process.  It  covers  two  broad  activities: 

"Determining  what  to  Procure. 

— Determing  how  to  Procure  it. 


JJ 


SLIDE  17 

The  Budget  and  impoundaent  Control 

Act  of  1974  created  the  Budget  CoMittees  of  the  Congress  and  the  Congressional 
Budget  Office  and  made  a third  key  committee  of  each  house  to  oversee  the  DOO  budget. 

This  congressional  budget  structure  has  the  effect  of  further  implementing  the  rec(«nda- 
tions  of  the  Procurement  Comiissioii  because  in  January  1978  the  President  will  present  a 
mission  oriented  bu^et.  ^ . 

This  has  increased  the  trend  in  the  authorization  and  appropriation  coMiittces  to 
move  toward  mission  area  consideration  of  the  Defense  budget. 

More  recently  we  have  begun  .'eviewing  groups  of  related  systems  in  a mission  area 
context.  - . “ 
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How  about  Identification  and  Definition  Phase? 
SllDC  ZZ 

This  new  phase  Includes: 


•' o(v>T 

..  /.  •^-  >P*.  .-..Vti^. 

and  Definition  Phace?  .^•'- 

■•■.:■  V-  ■■> ' 


^ gMlyzIna  the  agenor's  alsslon  In  relation  to  national  needs.  = ^ .(*{3  %c'S >-  / ' \ 

• geconclllatlon  of  agency  conponent  roles  and  Missions.  • .i,  ' A-' 


Intelligence  collection,  processing,  and  reporting. 


It  by  those  who  mst  face  the  threat  and  counter 

t y an  analysis  organization  reporting  to  the  agency  bead. 


I^oloax  assesswent  and  the  posh  of  technology  relative  to  the  pull  of  the  threat 
Assessiwent  of  existing  resources  and  capabilities. 


v.r:  i’' 


which  my  be  Invested.  ’ “tablish  the  general  u?  Itude  of  resources 


!^!i  ****  "^ss^O"  needs  which  comprise  the  POH  will  be  soeclflcallv  ittann 
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Something  new  Is  added  to  the  Conceptual  phase  as  well.  Approval  of  ^ mission 
need  grants  authority  to  explore  alternative  methods  of  satisfying  the  need  with  emphasis 
on  generating  Innovation  and  conceptual  co^ietltlon  from  Industry!  Toamorrow  we  may  get 
soi»  Insight  from  Industry  and  OFPP  as  to  how  this  will  Inpact  the  process.  f 

"a 

Trade*off  studies  of  cost,  schedule  and  perforaance  becoae  aore  coaplex  and 


laportant 


Vim 


f 'A. 


Risk  analysis  of  new  concepts  or  unproven  tMhnology  combines  subjective  Jud^ent 


with  quantiutive  methods 


Vi 


'f  / 
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Economic  analysis  of  the  alternative  concepts— over  the  life  of  the  concept  and  In 
relation  to  othw  W>ITt1cal  and  economic  goals— becomes  critical. 

I was  happy  to  see  the  tervs  acquisition  jwd  procurement  used  Interchangeably  In 
the  program  and  discussions.  • ^ ^ 

If  you  are  a procurement  purist  you  may  decide  that  procurement  researeh  only 
concerns  Itself  with  the  letting  of  contracts  for  obtaining  alternative  solutions. 

. CM)  however  Is  responsible  for  the  application  of  all  public  funds  and  we  must  , 
exmalne  the  very  complex  and  significant  operation  of  determining  what  to  procure. 

SLIDE  25  . C ' 

Joint  F1nanc1a1*16i  ^iPleader.  P.  L.  91-379, 

Aug  1970  esUbllshed  tht  CWt  JfccjWtlaj  Standards  Board.  The  C.  6.  Is  Chairman  of  the 
Board.  ..  ....  ? — v.  ‘ 

.y  , ,,  „ , 

* • Congress  has  not  seen  fit  to  specify  a Joint  Procurement  Improvement  Program  as 
’such.  OFPP,  the  FPI,  and  these  syiqiosla  I suppose  constitute  the  current  effort. 

J Would  It  benefit  from  being  Identified  as  a program?  j 

t ■ * ‘ N ^ * • ' * v ^ . • . • ^ s’  I * -S'.  0 . f 

SLIDE  26  . • ' ..  . ..  ..  . . ' f 
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If  so,  where  Is  the  overall  Hanagamnt  Improvement  Program  It  support*?  ; 
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• AN  INDEPENDENT  AGENCY  01  THE  LECISUTfVE  BRANCH  OF 
‘ THE  GOVERNKENT 

O CREATED  IN  1921  BY  THE  BUDCH  AND  ACCOUNTING  ACT 


O HEADED  BY  THE  COMPTROLLER  GENERAL' AND  THE  DEPUTY 
COMPTROLLER  GENERAL,  EACH  WITH  15  YEAR  TERMS 

• ELMER  B.  STARTS  • APPOINTED  IN  1986 
AS  FIFTH  COMPTROLLER  GENERAL 

• ROBERT  F.  KEUER  • APPOINTED  IN  1969 
AS  DEPUTY  COMPTROLLER  GOfERAL 

• size  . ABOUT  5,500  EMPLOYER 


• LOCATION  - HEADQUARTERS  IN  I WASHINGTON,  D.G. 

ICE^ 


SUM  t 


REGIONAL  OmCES  IN  15  MAJOR  CITIES 
OVERSEAS  OFnCES  IN  FRANKFURT,  HONOLULU, 
BANGKOK  AND  PANAMA 
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AUTHORITY  IS  BROAD 


"THE  COMPTROlUR  GENERAL  SHAH  l»iVESTIGATE....A!i  • 

1 MAHERS  REUTING  TO  THE  RECEIPT,  DISBURSEMENT, 

‘ AND  APPLICATION  OF  PUBUC  FUNDS."  (1921  ACT) 

"THE  COMPTROLLER  GENERAL  SHALL  REVIEW  AND  EVALUATE 
THE  RESULTS  OF  GOVERNMENT  PROGRAMS  AND  ACTIVITIES... 
(CONGREWiONAL  BOOGH  AND  IMPOUNCMENT  CONTROL  ACT 
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BtSI  AVAIIABLE  COPY 
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;iiin  JT  . ^ ■ 


CATEGORIES  OF  AUDIT 


HNANCIAL  ACCOUMTABIUTY 

• PROraiETY,  ACCURACY,  AND  LEGALITY  OF  RNAHCIAL  TIt\NSACTIONS 

• SETTLEMENT  OF  PUBLIC  ACCOUNTS 

• RELIABILITY  OF  FINANCIAL  STATEMENTS  AND  REPORTS 

• ADEQUACY  OF  ACCOUNTING  SYSTEMS 


MANAOLRIAL  ACCOUNTABILITY 
• EFFICIENCY  AND  ECONOMY  OF  OPERATIONS 


f 


PROGRAM  ACCOUNTABIUn  - 1. 

• EVALUATION  OF  PROGRAM  BENEFITS,  RESULTS,  OR  AGCOMPUSHKENTS 

• AND  ACHIEVEMENT  OF  INTENDED  OBJECTIVES 
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NIT  It  im 


>CC0UNTA!fTS  AND  AUDITORS  ' 

‘ *,643 

BUSINESS  ADMINISTRATiON/OSANAGCMENT 

009 

ECONOMISTS  AND  OTHE^  SOCIAl  SCIENTIST! 

. w 

AnOHNETS 

129 

ACTUARIES  AND  OTHER  MATH  SCIENTISTS 

. 69 

CUIMS  ADJUDICATORS 

36 

COMPUTER  AND  INFORMATION  SPECIAUSTS 

36 

PERSONNa  MANAGEMENT  ^CIAUSTS 

26 

Tl  ■ ^ 

ENGINEERS 

29 

TRANSPORTATION  SPECIAUSTS 

5 

AU  OTHER 

110 
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SOURCES  OF  IKFORK'r’T.vn  ABOUT  GAO  REPORTS 
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• MONTOLY  UST  OF  C.*J  v»C!lT5 
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•UST  OF  GAO  PUBUCATIONS  (SEMI-ANNUAU 

• CONGRESSIONAL  RECORD 

• DAILY  UST  OF  COMPTROLLER  GENUAL  REPORTS 

• MONTHLY  UST  OF  OAO  REPORTS 

• REFERENCES  TO  SPECIFIC  REPORTS  BY  MEMBERS 

' f 

• ANNUAL  REPORT  OF  COMPTROLLER  GENERAL 
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CONGRESS  LOOKS  AT  DEFENSE 
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Budget 

Committees 
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Armed  Services 
Committees 


MISSION  STUDIES 


GAO  does  not: 


Define  nissions.  . 

Make  Inteliigence  esGmetee. 

Suggest  mintanr  policy,  strategy,  er  tactics. 

-Specify  operatioiis  requirements  for  weapon  systems. 
Suggest  appropriate  spending  levels.- 


GAO  does: 
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State  laission  reqairerocnts  as  defined  ky  OSD. 

Describe  the  enemy  threat,  using  DOD  hteliigenee  date  and 
available  threat  sssessment 

Describe  current  Inventory  ef  weapon  systems  using  available' 
DOD  performance  data  and  costs. 

Describe  current  and  planned  R&D  programs,  purposes, 
end  projected  costs. 
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PROCUREMENT  RESEARCH; 

I5  MRe  owe 'BEST  WA**? 


Major  Eyle  U.  Lockwood 
Deputy  Director 

Air  Force  Business  Research  Management  Cen:^r 

Lt  Col  Daniel  E.  Str^r 
Executive  Director 

Air  Force  Business  Research  Management  CCntwr 


IMTHODOCTIO:. . 

. ' ' f 

• The  release  of  tne  findings  of  the  Ccmaiission  on  Govem- 
oent  Procureaent,  the  establishment  of  the  DOD  Procurement 
Research  Symposium,  and  formation  of  the  Federal  Procurement 
Institute  within  the  Office  of  Federal  Procurement  Policy 
have  dram  attention  to  the  sabject  of  procurement  research. 
These  events,  by  implication,  have  stimulated  a search  for  : 
better  definition  of  both  procurement  and  procurement  research. 

Thus,  there  are  attempts  to  define  the  boundaries  of  the 
process  which  we  call  procurement  and  to  define  the  process 
by  which  research  of  the  procurement  process  is  conducted. 

One  paper,  "A  Proposed  Definition  and  Taxonomy  for  Procure- 
ment Research  in  the  Department  of  Defense,"  atrempts 
to  define  the  procurement  research  process.  Another  approach 
to  defining  the  procurement  research  process  is  being  under- 
taV.en  by  an  eleinent  of  the  General  Accounting  Office  (GAO). 

In  both  of  these  endeavors  the  implication  could  be  drawn 
that  the  result  will  reflect  the  one  best  concept,  the  one 
best  definition,  and  thus  the  one  best  way  for  procurement 
research. 

The  purpose  of  this  paper  is  to  present  some  observations 
about  the  ;T.oces  and  diiner.sions  of  procurenent  research  which 
suggest  that  the  adage  "it  all  depends"  may  be  applicable  to 
procurement  research.  In  short,  there  may  not  be  "one  best 
way"  for  procurement  research. 

Procurement  is  largely  an  heuristic  process.  A variety 
of  rules  devis^  from  a variety  of  sources,  using  a variety 
of  techniques  to  satisfy  a variety  of  objectives,  are  used 
to  influence  the  structuring  and  performance  of  the  procure- 
ment process.  Traditional  public  elements,  i.e..  Legisla- 
tive, Executive,  and  Judicial  Dnnehes  of  Government,  using 
a spectrum  of  procurement  research  techniques  promulgate 
changes  in  law,  policy,  and  precedent  which  substantially 
alter  the  procurement  processes  operation. 


Do^'l’*^^i“i!'  ‘>y  the  GAO,  public  hearings  oe%TO- 

Other  processes  of  the  public  policy 
formulation  process.  The  Executive  Branch  influences 

through  the  promulgation  of  procur^eant 
Md  acquisition  directives,  instructions,  regulations  and 
equals.  Stales  and  research  projects  often  support^this 
a«  the  llsdt.  o£ 

L ^ **  * result  of  procurement  research  per- 

contractors,  and  specificallv  chap- 
tered ad  ooc  study  teams.  The  Judicial  Bra^h  influences 
process  through  decisions  made  after  studv 
P'ttinent  to  the  case  uJde?^- 
of  these  are  influenced  by  procurement  research 
done  by  private  elements  involved  in  the  procurement  prSPPS? 

4-  Contactors  may  enq>loy  procurement  research  to  qenerate 
improved  procurement  methods  for  their  own  activities  oP  2 
Procureront  policy  generation  process.  Academic  insti- 
tutions conduct  procurement  research  for  either  the  public 
or  private  sactocs  or  to  satisey  their  o«, 

sSdii.  associations  also  conduct' 

tidies  of  the  procurement  process  in  response  to  •^eauest- 
by  eitner  the  private  or  public  sectors.  ^equesto 

Although  the  reader  is  challenged  to  think  about  th- 

SseSS  elements  places  upon  p^ocSJ^ment 

research  to  sct-isfy  their  respective  objectives,  we  will 

dimensions  of  procurement 

Iv  Department  of  Defense  (DOD) . 

we  Ian  dimensions 

we  can  xlluroina^e  the  complexity  of  the  problem. 


PROCUREMENT  RESEARCH  MODES  ^ ^ 

oer'ST^r,!^  defined  as  a manner  of  doing  something  or  of 
® function  or  activity.  Using  this 

modes  of  procurement  research  ares 
/of  (2)  the  form  of  the  output  products** 

and  (3)  the  metaodology  of  the  study.  * 


Participants  ■ ‘ 

in  procurement  research  are  initiators 
researchers,  data  sources,  and  recipients  or  TOnsuaers.'  dod 
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procurenent  research  can  be  initiated  as  a response  to  exter- 
nal sources  such  as  Congress,  the  GAO,  or  higher  levels  wi^in 
' Executive  Branch.  Agency  decision  nakers  can  also  be 
initiators  of  procureoient  research.  These  aay  be  procure- 
nent officers;  Directors  of  Procurenent;  and  pr.-Hnireoent 
stAff  elements.  On  occasion,  researchers  independently 
initiate  proctirenent  research.  Each  of  the  participants  nay 
problems  within  the  procurement  process  which,  with 
or  without  acceptance  and  approval  by  the  other  participants, 
become  the  basis  for  initiating  procurement  research. 

to  searchers  such  as  ad  hoc  teans,  academicians,  and  even 
staff  elements  also  participate  in  the  procurement  research 
process,  tosearclters  may  act  individually  or  as  organized 
continuing  teams  in  the  procurement  management  review  sense. 
Researchers  may  also  be  organized  as  ad  hoc  teams  from 
responsible  staff  elements.  Such  specially  chartered  ad  hoc 
interdlscipluury  teams  are  frequent  participants  in  procure- 
ment research.  ^ *-?  ' 

.r 

Students,  academicians,  and  professional  researchers 
also  participate  in  procurement  research.  Their  activities 
nay  be  gratuitous;  that  is,  they  are  interested  in  the  pro- 
curement process  and  are  conducting  research  and  study  into 
the  procurement  process  to  satisfy  their  personal  quest  for 
knowledge.  Professional  researchers  may  also  be  involved. 

Data  sources  also  participate  in  the  procurement  research 
process,  l-iany  procureiaent  research  findings  result  from 
interviews  and  questionnaires  supplied  by  procurement  prac- 
titioners. Other  data  sources  may  be  operators  of  data 
gathering  systems  which  collect  information  on  the  procure- 
ment process.  Contractor  personnel  are  also  participants 
as  data  sources  which  are  important  in  the  conduct  of  pro- 
curement research. 

Recipients  (or  consumers)  of  procurement  research  may 
or  may  not  be  the  same  set  of  people  who  initiated  the  pro- 
curenent research.  Decision  makers;  staff  elements;  and 
more  often  than  not,  the  library,  the  files,  and  the  desk 
drawers  are  the  usual  recipients  and  the  repositories  for 
procurement  research. 

Thus  initiators,  researchers,  data  sources,  and  recipients 
or  consumers  nay  be,  and  frequently  are,  the  sane  people  or 
elements.  They  are  just  wearing  different  hats  at  different 
tines.  Yet,  each  carries  a different  jserception  of  the 
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research  need,  approach,  data  revjuiraiaents,  and  utilization 

circumstances  of  the  research  ‘ 
would  be  surprising,  therefore,  if  any  consensus 

on  4e  pH'uc!- 

• • * * r 

Indeed,  given  the  wide  disparity  of  backgrounds  oo-aant. 
zational  loyalties,  and  objectives,  it  would  perhaos  be 
Wisest  to  expect  diversity  when  the  participant’s  view  of 
procurenene  ranearch  is  solicited.  AgreeneST on  a "testi 
approach  would  seem  unlikely.  ^ - 

Form  of  Output  Product 

The  fora  of  the  output  is  also  a mode  of  procurenent 
research.  What  is  desired  has  great  impact  on  wh^J^ho^ld 

^v  ’accomplished.  oStput  p2i^uc^s 

^y  be  one  or  more  of  the  followings  a foraal  report 
decision,  a regulation,  or  a computer  program.  * 

shviPf!!^^  reports  are  usually  documented  in  a rigid  academic 

scientific  method.  The  objective 
i-  disclosure  of  facts  and  methodology  establishing  a 

basis  for  replication.  Other  formal  reports  marii  S2ition 
Eiders  for  use  by  the  staff  elements  or  decisioJi  mkHs 

formalized  supporting  docun^n^a- 
hi  2*.»  Another  fora  of  procurement  research  output  oav  simply 
* |Sci3ion  to  choose  one  course  of  action  ovtr  another.* 

The  decision  izay  be  reflected  in  a regulation,  dir<*ctiv»  o*- 
instruction  formulated  from  the  findings  develooed  tiirough 
procurement  rasearch  activities.  This  cons titu2S  iSSlu- 
tionalization  of  the  research  results,  in  other  cases  a 
y^uter  program  is  a product  resulting  from  procurement 

continued  utilisation  constitutes  ija?lemen+-a- 
tion.  Outputs  of  the  research  process,  like  its  pa-^icloa-ihs 
n22s  several  foms  a.aJ  satisfies  widely  divergent  ne**ds^ 

^th  tae  uetaodology  and  the  ground  rules  impact  on  the 

should  be  managed  and  how  its  nanace-* 
ment  snould  operate.  ' 

Mathodoloc-/ 

-npiov2;‘'®‘.3'“  procurement  research  is  the  n:ethodolo«c, 
^mployed.  iVo  general  categories  of  methodology  are  quan-* 
ticati/e  and  gaalitative,  >3jantitative  riethodologies^usual  i-r 
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deal  with  issues  where  aeasurable  data  are  present  and  the  ’ ~ 
tools  of  operations  research  and  panagenent* science  can  be 
applied,  often  resulting  in  the  use  of  a hypothesis  tost.  j 

Qjalitative»tyoe  wethodologies  draw  sore  upon  the  behavioral  ; 
sciences,  are  descriptive  in  nature,  and  often  use  the  1 

research  question  approach.  Of  course,  combined  nethodol-  ' I 
o-Ties  nay  be  used  in  situations  dealing  with  soiae  subject  i 

which  require  soine  rigorous  subjectivity  in  their  analysis.  » 


Modes  or  manners  of  performing  a particular  function,  in  I 
this  case  procurement  research,  all  operate  within  certain  J 

specified  ground  rules,  constraints,  or  operating  envelopes.  t 
These  ground  rules  both  limit  the  outcomes  flexibility  i 

and  provide  focus  and  direction  for  study.  Soife  of  these  ? 

ground 'rules  for  procurement  research  are:  compliance  review,  ^ 
evaluation,  or  improvement  and  innovation.  The  compliance  1 

mode  of  procurement  research  would  be  used  to  determine  ' 

compliance  with  an  existing  regulation,  directive,  or  speci- 
fied practice.  A simple  hypothesis  test  might  be  used.  The  ' ' v ' - 
evaluation  mode  night  be  applied  to  evaluate  the  efficacy  of 
the  procuremnt  process  to  determine  if  a change  i's  warranted. 

This  might  involve  the  use  of  both  quantitative  and  quali- 
tative techniques'.  j ■ 

The  remaining  mode  may  also  be  used  to  seek  improve- 
ment or  innovation  within  the  procurement  process  itself. 

In  contrast  with  the  compliance  and  evaluation  modes  which 
suggest  operation  within  the  limits  of  regulations,  the 
last  case,  in^rovement  or  innovation,  suggests  that  regu- 
lations and  formal  authority  patterns  should  not  be  viewed 
as  constraints  by  the  researcher.  As  we  get  further  from 
defining  the  manner  in  which  procurement  research  is  con- 
ducted and  begin  to  define  factors  which  influence  our  view 
of  procurement  research,  the  dimensions  of  procurement 
research  come  into  play.  * ^ 


procure:<ent  resea2ce  di:ensio?is 

Dimension  is  defined  as  the  set  of  circumstances  or  envi- 
ronmental factors  within  which  something  is  vie:«ed.  Some  of  - - 
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the  dinenaions  which  are  used  to  view  procurcKnent  research 
are  the  flow,  the  outcome  level,  the  breadth  of  apalicatian, 
and  the  objectivity  of  the  data. 


Sequential  Flow 


. t A 


' t ^ .V  yi 


toe  ditaenaion  of  procurement  research  which  attracts 
considerable  attention  is  the  sequential  flow  dinension. 
this  starts  with  the  definition  of  need,  followed  by  the 
initiation  of  the  proiect.  the  conduct  of  the  research,  the 
docusien cation  and  presentation  o:  research. results,  tne 

acceptance  and  iaplgcentati^n  o:  those  reajtlcs.  and  the 
follow— ^p  of  ^e  procuretaent  research.  This  flow  is  siaiilar 
to  the  flow  of  the  acquisition  process  and  is  often  accos- 
panied  with  an  identification  of  the  decision  makers  who  < 
have  the  authority  to  commit  resources  for  research. 


The  need  definition  step  usually  includes  defining 
the  problem  and  planning  the  study.  This  is  followed  by  a 
formal  research  initiation  approval  validating  the  research 
need  and  priority.  At  ter  the  reseaurch  project  has  been 
initiated,  the  research  is  conducted  including  actions  which 
select  the  researcher,  the  methcKioiocjy,  and  identifv  and 
obtain  the  necessary  data.  These  steas  are  followed  by 
Uoc«rTent3tion  of  the  study  and  presentation  of  the  results. 
Following  the  documentation  and  presentation  step  is  the 
.decision  to  accept  the  results  a.nd/or  inplenert  the  recers- 
mended  actions.  The  inplementatior*  ste-?  should  be  followed 
by  a formal  process  where  the  results  oc  the-inplsmentati^. 
are  evaluated  and  possibly  a new  research  project  defined 
a.'id  the  flow  reinitiated. 


The  sequential  view  considers  each  use  of  research 
to  be  equally  likely,  ntjually  useful,  '-ow.iver,  social 
science  has  evolved  additional  vie*#s  of  the  research  orocess 
wliich  should  be  drown  upon  in  formulating  the  rules  for  n«- 
caading.  Ilelastadter  i.n  "research  Concepts  in  llarian  Behavior" 
orfers  a construct  consisting  of  three  dimensions:  outcome 
level,  breadth  of  a?plic.ation,  and  data  objectivity,  (Fi.-r*ire 
• "^is  constf.ict,  which  affects  each  research  effort,” 
provides  three  more  dimensions  of  procurement  research. 
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Acron  senna  APrwk  n» 
8REM)TH  OF  APPLtCATVN 


•aa^SW 

’'rtfoicTivc  ■ 

'OCSUITTNE 

UVEL  OF  OUTCOME 


Outcome  Level  . ' -L  - - • 

This  diaension  addresses  the  notive  for  the  procurement 
research  project  in  terms  of  the  outcone  desired.  If  the 
outcome  level  is  intended  to  be  descri.>tive  in  nature,  the 
purpose  of  the  project  would  be  to  define  what  the  world 
looks  like.  These  may  be  statistically  based  presentations 
or  qualitative  descriptions  of  the  environment  in  which 
procurement  operates  and  usually  includes  a rigorous  defini- 
tion of  the" process. 

/mother  aspect  of  the  outcome  level  dimension  would  be 
a study  which  permits  prediction  of  iaportant  characteristics. 
Hera  the  research  is  directed  toward  searching  for  relation- 
ships among  variables  which  have  sonie  high  degree  of  cer- 
tainty in  tarns  of  their  ability  to  predict  the  future. 
Excucples  of  these  night  be  the  construction  of  parametric 
cost  models. 

Another  desired  outcome  night  be  diagnostic  research 
results  seeking  to  offer  both  prediction  and  understanding 
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of  why  certain  rc)aLloashit.s  exi-Jt;  that  ir,  why  cVcs  a 
leaniir.ij  curve  opt-rato,  or  why  Joes  a parariotric  cort  wod«»'’ 
work, 

' f !'>■}■ 

Within  each  of  these  diraensions,  the  desire  nay  be  to 
^uwtc.nd  the  proctss  or  subject,  c2«.rifv  the  procesi^  cr 
• ..iLa.owe  c chanco  to  tne  process  or  ciecTslo;!  rule.  For 
txaaplc,  one  coujd  undertake  a study  of  learninc  corves  to 
seex*  bettor  understanaing  as  to  why  t!ic'*  predict  co«-t  at 
tne  levels  that  they  arc  predicting.  oTio  could  also  under-  • 

^ ®tudy  to  seeh  clarification  of  the  aoplication  of 
tiic  learning  curve  to  a particular  situation,  or  one  could 
unoertar;e  a study  which  would  support  changing  the  learning 
curve  Bodel  being  used.  . , ^ 
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Breadth  of  /.pplication  * 
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^aiotner  dimension  of  procurement  research,  also  drawn 
fron  Kolmstadter,  is  the  breadth  of  application.  Once  the 
re£uj.ts  of  pror-’reir.ent  research  arc  received,  the  cuestion 
j-s  an  which  are.*a  will  they  be  applied.  Kelmstacter  iden- 
tifies lour  oasxc  categories;  action  research-  service 
Research,  applied  research,  anc  pu;<  or  has IF  research. 

can  be  argued  to'^not  be  callec  research  at 
a-i.  The  initiator  has  no  intention  of  yielding  a research 
prtwuct;  but  intends,  like  the  Hawthorne  effect,  to  gain 
added  efficiency  and  effectiveness  by  drawing  attention  to 
® ?®2rvicular  subject  through  the  conduct  of  a study  or 
research  project. 

Service  research  is  usually  done  at  the  recuest  of  a 
particular  crgani ration  or  decision  maker,  and 'the  results 
ere  often  applicable  only  to  that  particular  decision  maker 
or  setting. 

Applied  research,  on  the  other  hand,  is  intended  to 
yielu  findings  ana  results  which  can  be  used  within  a riven 
field  or  body  of  know*ledge;  and  pure  or  basic  research  es 
'^''•''-^rtalzen  with  the  rtotivr  of  seeninr  bettor  undersiendi-'t 
and  knowledge  of  a civen  ■'-  ■•••  ‘ 

the  results'is  intended. 


ifild.  Ko  direct  application  o 


_v:ithin  each  of  these  categories,  one  could  define  on 
impj euen tati cn  targe t for  procurement  research;  that  is. 
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a stud-/  could  be  driving  toward  evaluating  an  asisting  -.olicv; 

that  ia,  a high  leval  dacision  rule  which  notivatas  the^ 

uxocureaent  process,  a proced«iro  which  nay  also  be  at  a 
relatively  high  level  of  the  procurenent  process,  but  x^hich 
guides  how  the  policy  will  be  inplenented.  Or  the  study 
CJUla  be  targeted  to  the  operation,  the  perfocaance  of  pro- 
curcaient  practitioners  in  their  day-to-day  operations. 


Ohiectivity  of  Data 

I 

The  last  draension  of  research  which  lay  be  used  to 
procurement  researcii^  is  the  obiectivitv  of  the  data 
which  went  into  the  study.  Helms tad ter  idea tii las  three' 
characteris^cs  of  data  objectivity  as  being:  libraxr/ 
ressaren,  field  research,  or  laboratory  research.  Librnrv 
research  relxes  neavily  upon  what  other  researchers  have 
p-ablished  in  previous  studies,  learned  journals,  periodi- 
cals, and  textbooks.  Field  research  suggests  that  in  cany 
cases  the  researcher  can  only  control  tha  questions  that  he 
nay  ask  and  the  tining  and  location  in  which  he  will  ask 
those  questions.  In  this  case  the  sampling  method  controls 
the  objectivity  of  the  data  which  were  used  for  the  studv. 
Laboratory  research  depends  upon  controlling  input  and  output 
parameters.  In  the  field  of  procurenent,  laboratory  studies 
vhild  be  usually  liisdtad  to  slaulation  no^^els. 


rhoc’Jkz:!^:*^  ?esea.ic:  TnAcs-opps 


The  ;X>cIe3  of  procurenent  research,  such  as  the'partici- 
pa.tts,  the  form  of  the  product;  and  the  dimensions  of  pro- 
curanent  research  such  as  the  breadth  of  application,  the 
level  of  outcore,  and  the  objectivity  of  the  data;  wemits 
asking  the  question,  fro.a  the  decision  maker's  point  of  view, 
when  can  I get  results?  It  also  suggests  that  the  initiator 
or  consumer  of  procuranent  research  nay  have  to  nake  sone 


tra.'!e-oif  between  a rigor  index  v;hich  would  be  composed  of 
combining  such  factors  as  the  data  objectivity,  the  breadth 
of  npnllc.ition,  and  the  level  cf  outcome  desire-i;  against 
t!:e  ti.ie  frame  ivhen  he  might  expect  results  in  one  of  the 
research  p'roduct  foms  which  we  have  discussed,  (Figure  2) 
'fills  construct  can  be  employed  by  the  .Incision  naker  who 


to  be  conduces.;  and  when  he  might  expect  to  get  results 
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PRINCIPLES  FOR  IMPLE'lZrrrATION 


Thd3-  constx-uc^:';  of  r.ode3  and  Jinensior.^  of  procu^■»^.f^a^ 
ro39«irch  s»!ygest  f.vat  thero  is  no  ona  best  way  for  procurs- 
r?ant  rsseircli.  Acaueraic  argunent  night  pursue  the  nerits” 
o-  a^ch  ilitaension  and  noda;  however,  this  will  not  helo 
decision  makers,  researchers,  and  operators  cotae  to ’arias 
vtch  tae  oast  way  to  utilize  procurement  research  to  *.  enera' 
ac.ii*.*rfao,.2  ond  long-st;indiag  iaprove.ierts  to  th«  {>ro'''i-eM,*n‘ 
process.  To  that  end,  there  are  five  principles  which’\^  ' 
J^sJa^h  aither  dimensions  or  r.o^’.es  of  procurement 
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Ihcsja  princiiJles  are:  • i.  • UiO. 

. >■'  * ' “ I /.A:  ' 

!•  [Secure  operator  and  researcher  ogreeneat^aad 
accsptonce  of  the  problsa  ilefinitioa  and  its  delimitation, 

-»■•■.  ■ f ■ -^tvp  " r' 

, 2.  Establish  an  input  kr.owladye  base  for  the. 

research,  . • y , . c;  i , 

" i'  ‘ --"j  ‘^"'•1  •:  < 

• 3*  Ksbahlish  an  output  fact  picture,  ^ 

" '•  ’ . ■•'  c r...  ’ .'i!.'--j' 

« .'>■*•  Satisfy  customers  of  procurenent  research,  and 

e -v.^  r . • •.jr.T’jr  .:t 

» 5 • CeAtAne  i*n  impXerientatxon  tool,  *.*  / **'*1^  ■ fO“* 

- - • ( ‘ ^ ^ ' .■r 


'Hie  first  prlJMSlple,  securino  acce3taiie.  <iiMi  ■mianwni 
on  the  problem  def^gtion-.W 


STtaH^ 

irabUSK 


both  iniciator  -ancGinsearcha,  to  cstabUiK  a com^non  bond 
and  coi&mon  frame  of  reference  for  the  project  itself.  This 
process  is  very  nuch  analogous  to  the  procurement  process 
itself,  where  the  customer  and  the  producer  come  to  grius 
with  what  Is  being  bought  and  why.  ‘witho«it  this  essential 
step,  procurement  research  nay  take  us  to  the  wro.ng  targec. 


ror 


Entabl  I'thment  of  t.ie  inout  knovled<7e  base  is  not  unlike 
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t.iH  prooifrfw  :»OA*/2.a^  ccjcaatuau  proposed  b*/  l\eQaer  and  T’^eiTO-^ 
in  their  book.  The  ^Mna-er>  ::o‘need  to  know  what 

we  know,  and  we  need  to  know  what  we  don’t  know.  The  estab- 
l..s.u3er.t  of  the  inp»it  knowtedgu  base  nay  Jis  a research  orojeej 
^tselt.  ilcwever,  unless  we  can  establish  the  historic  a.ad 
present  ft-ict  picture  for  what  the  current  problA»n  is,  results 
rw/  be  juastior.ahle  or  'joft.  Without  facts  on  the  problem’s 
inipact  on  our  TOasures  of  nerit,  i/e  cannot  decide  where  to 
apply  management  and  research  emphasis  and  resources. 


At  the  conclusion  of  the  research,  the  outont  *uct 
:riaku£2  =^usfc  be  carefully  *;«tablished  a':^  G:a  itr  JG ».  which 


( 


U'-T--  '■  r ' - e :a  wnich 

Oj.ten  in/olves  going  back  to  the  in^  ut  :novl«jr!ga  base.  The' 
research  results  must  be  defined  and  prasented  in  tha  same 


^ - ••  ^ ^ i Cs.srci*  w<;\*  Jill  Ulls  SUi«>S 

^ann-r  in  which  the  probl3r»  was  original!.-/  defined  and  the 
iii.rjc  !:r.o^vIijdge  b was  r-otnoliji-?:!.  A n.hLfting  fraa-e  o: 
refer-.mee  hel^^s  neither  ro^earchtir  nor  o;>erator. 


•Ceoorng  c;into^>er  si tiofaction  pnrarr.ount  roguireo  strong 
ai:d  frch;»;ent  intarnctiO:i  liatw*»^n  rnseircher  and  operator  . 
duri:i j all  sfcjpa  of  rhe  r'^'search  i-»rocnjij  a!id  tiio  ahiiiti*  to 
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be  flexible  and  to  make  necessary  trade-offs  so  that  the 
results  will  be  ready  when  the  opportunity  for  their  use 
arises.  Tiie  decision  laaker  riust  be  flexible  enough  to 
interact  with  the  researcher,  to  accept  redefinition  of  the 
problem,  and  to  adjust  his  expectation  level  to  that  which 

to  produce.  TIte  researcher  must 
O’-*  enough  to  adjust  methodologies,  accept  problora 

redefinitions,  and  occassionally  provide  "satisficing*  (i.e 
good  enough  for  the  purpose  vs,  optioun)  responses.  * 

Definition  of  an  implerontation  tool  is  the  last,  and 
perhaps  most  significant,  principle  which  must  be  borne  in 
ini.-id.  When  procurement  research  is  undertaken  with  the 
objertive  of  securing  implementation,  the  researcher  and 
tne  initiator  must  mutually  determine  the  most  practical 
and  effective  way  to  implement  the  research  results.  Will 
ii\alementation  consist  of  a regulation,  a directive,  a • 
computer  program,  or  a training  program?  Who  will  be 
®“^ooted  and  in  what  manner?  If  an  implementation  tool 
has  not  been  defined,  then  the  results  nay  sit  idle  on 
the  shelves  of  the  library  or  in  the  lower  left-hand  desk 


drawer 


COnCLUSION 


We  have  presented  in  this  paper  a construct  of  modes  and 
dimensions  designeti  co  clarify  the  many  forms  taken  by  pro- 
curement research.  Employing  the  construct,  the  paper  addressed 
the  question,  procurement  research — is  there  one  best  way?, 
offering  tlie  answer:  "it  all  depends  on  what  mode  and 
dimension  the  problem,  and  the  appropriate  research  plan, 
presents  to  management  and  researcher  alike."  Only  in  the 
context  of  the-problera  and  its  environment  can  we  decide  on 
the  best  way  for  procurement  research  to  proceed-. 
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Introduction 


If  one  studies  inodera  Industrial  processes,  concentrating  particularly  w those 
which  Include  frequent  changes  .In  goals,  budget  •rollercoastering,"  and  a 
of  design  and  production  complexity,  he  finds  few  that  compare  with  the  acquisition  of 
major  military  systems.  It  Is  certainly  doubtful  whether  any  organization  or  in^stvy 
faces  more  baffling,  frustrating  challenges  than  those  encotmtered  routinely  by  DOU 
(and  associated  Industry)  scientists,  engineers,  technicians  and  mmgers;  they 
contend  almost  dally  with  schedule  slides,  performance  failures  and  dollar  escalation. 


L.' 


.1 


The  dollar  airounts  are  substantial.  The  USAF  FY  78  and  FT  79  RDTAE  requests  for 
general  purpose  forces  were  964.5  million  and  980.8  million  respectively.^  The 
requesU  (sa«  years)  of  the  three  services  for  Strategic  Forces  were  1.454  billion 
and  2.077  billion  resoectlvelv. ^ The  Navy  listed  aircraft  Program  Acquisition  costs 


and”2r077' bill Ion  respectively. ^ The  Navy  listed  aircraft  Program  Acquisition  costs 
of  2.204  billion  In  FY  1977,  while  the  Amy  showed  1.748  billion  in  FY  1976  for 
Aircraft  Missiles,  Tracked  Combat  Vehicles  and  other.9 


( 


However,  the  size  of  the  investment,  even  if  described  in  terms  of  scarce  dollars. 
Irretrievable  time  or  wasted  efforts  of  thousands  of  dedicated  individuals,  is  not 
the  crucial  element;  while  size  does  contribute,  the  fundamental  reason  for  the 
management,  technical  and  economic  difficulties  is  change— change  in  unit  performance 
(actual,  estimated  and  desired),  change  in  system  goals,  change  in  budget  authority 
and  change  in  quantities  desired.  The  Aerospace  Daily  of  February  24,  1977,  carried 
these  1 terns : 


1 


ROTiE  - Research,  Development,  Test  and  Evaluation. 
Aerospace  Dally,  February  24,  1977,  Page  293. 
^Aerospace  Dally,  January  19,  1977,  Pages  98-99. 
‘ibid. 
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^ if®  Minuteman  II  alsslles  covered  by  the  Ford  Adelnlstratlon 

ss'Ki  *• 

-bililm^  ti^av’^KiT??  ^®®  •ilHon  In  FY  '78  — froo  $2.15 

78  ' •****  * '••t*  ^08  • y*»r  to 

orfolMl  Miif  '^“’‘*"9  **»<'»9  would  be  applied  to  the 
or  ginal  $1.76  billion  request  to  bring  It  down  to  $1.43  billion.” 
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"The  Sikoisky  CH-53  helicopter  for  the  Navy  and  Karin#  Chm<  bn^c 

2*JartU^*£y;^JJ^iiteh  -in loneliest) 

« part  or  a aove  to  stretch  out  the  progran  by  a year." 

half— froi  $200  Million  to  $100 

2«  ^1flJSlI"?d2lHf!2  (Secretary  of  OefeJse)  Brown 

. vulirJbllltJ!- ^ l«»entlf1ed  as  a concern  over  the  helicopter's 

oJt  2lt1rel5'fOT‘'l"ll8r-in?*  8^s-e5“<PPed  CS6N  was  knocked 
Sllim  wsViItL  r ^ "niion  reduction  while  another  $43 
■niion  was  deleted  for  conversion  of  an  LSI  to  a phm  tii* 

PHN  prograa  will  be  further  reviewed.”  • PHH.  The  . ^ . 

JISd^nnn^JsesTr  SSoSS^JpJSJ!^^’:;  s^^l^e"1treSL^^ /.  i 

ScTr^JiJ* 

CongresSn  mu«  critiLe  DOD  Political  environment  In  which 

grans  be  awarded  to  connAni#^  in  while  at  the  same  tine  demanding  that  pro- 

wtlwaJeTf  tSe  iJt^?S^?e^su^i^^^„^^/‘*‘*^?^  districts;  and  he^  certain^ 
missiles,  aircraft  and  conputers  sophisticated 

thing;  b^ng  abl^to*expl^n"how**t^*tfit!Ii***'*  (*od  similar)  questions  Is  one 

would  probably  bj  S4st  to  it 

stand  how— and  why—  the  entire  ^ople  within  the  system  really  under- 

that  the  retsUF^^of  Jofllre  il  1*  "®*‘ 

i«?Jl?S;ioI?tJ  Sop  UP  . 
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These  and  other  perturbations  of  the  environment  do  not  surprise  the  experts. 

They  realize  that  what  they  {the  experts  and  practitioners,  that  Is)  need  are  tools  or 
techniques  or  "rules  of  thuiii)*  that  will  enable  them  to  assess  more  accurately  the 
Impact  of  changes  which  they  know  will  surely  come.  For  example*  what  will  be  the 
Impact  upon  a production  program  If  funds  are  cut  by  $10  million  or  $100  mllllom? 

How  will  a 10X  Increase  In  workers'  wages  Impact  upon  quantities  to  be  produced?  What 
should  be  the  expected  dollar  return  (to  000)  If  a production  schedule  Is  stretched 
out  and  monthly  deliveries  reduced?  If  system  performance  goals  are  Increased*  how 
many  additional  units  must  be  procured  - or  should  we  reduce  the  total  quantity  while 
Increasing  the  unit  performance  of  each  Item  remaining  to  be  built? 


Objective 


The  objective  of  this  paper  Is  to  propose  one  way  of  developing  a systematic 
understanding  of  the  Weapons  Acquisition  Process,  that  of  model  building  and  computer 
simulation.  A computer  simulation  and  modeling  approach,  such  as  we  propose,  may 
permit  managers  such  as  SPO's,  the  user  coMiand  requirements  elements,  military  depart- 
ment staff  agencies,  or  contractor  corporate  headquarters,  to  answer  such  questions 
from  wUhIn  their  own  resources;  the  model  is.  In  effect,  a planning  tool  to  be  used 
to  simulate  the  impacts  of  proposed  changes.  , . . 

The  remaining  parts  of  this  paper  briefly  describe  simulation  and  our  approach  to 
it  (System  Dynamics),  explain  the  rather  "simple-minded"  but  Illustrative  model  we 
have  chosen  to  demonstrate  how  this  approach  could  be  applied  to  the  weapon  system 
procurement  process.  Indicate  how  such  a model  can  be  used  to  gain  Insight  Into  how 
changes  affect  the  system,  and  discuss  how  our  model  could  be  expanded  and  enhanced  to 
suit  a real  weapon  system,  and  suggest  - In  our  conclusion  - potential  uses  of  such 
expanded  applications. 

Hethodoloqy 


Our  approach  In  this  paper  Is  that  of  simulation,  and,  more  specifically,  a version 
of  simulation  called  system  dynamics.  Simulation  has  been  around  a long  time  and 
mooelers  who  use  simulation  have  generally  followed  a two  step  process.  First,  they 
construct  a model  of  some  system  of  Interest  to  them  In  order  to  Imitate  the  system's 
behavior.  If  the  model  can  successfully  Imitate  the  system's  behavior  under  a wide 
range  of 'circumstances,  the  modeler  will  feel  confident  enough  to  move  on  to  the 
second  step:  running  policy  experiments.  In  this  step  the  modeler  alters  particular 
Inputs  to  the  model  that  correspond  to  real  world  policy  options.  For  example,  the 
Congress  could  cut  the  budget  of  a program  by  20X.  By  Interpreting  the  nxxlers 
responses  to  the  changes,  the  modeler  hopes  to  understand  how  the  real  system  would 
respond  to  the  same  policy  changes.  Once  a model  has  been  developed  and  confidence 
has  been  gained  In  Its  utility,  the  modeler  can  make  his  model  available  to  Interested 
users  (for  example,  a SPO  If  the  model  Is  one  of  a sjfstem  for  which  the  SPO  has 
responsibility).  The  user  then  can  test  out  real  system  reactions  to  policy  changes 
using  his  model  of  the  real  system  as  a surrogate  for  It. 


In  physical  systems,  simulations  are  relatively  common  and  run  the  ganxit  from 
simple  hand  simulations  with  table  top  models  and  phjrsical  mock-ups  to  more  recent 
and  exotic  mathematical  models  and  computer  simulations  such  as  those  used  to  control 
space  flights. 
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**P*'^*  •'*  "ow  *«''k  constructing  coaputer  based  slau- 
cardiovascular  systems,  and  so  on,  to  tS  new  medical  ‘ 
policies.  In  social  systems  (and  the  weapon  procurement  system  Is  a social  svstMl 

all^tld^tniiiini  ****  ***"  * increase  In  the  resoums  * 

1-  interdisciplinary  computer  simulation  models  of  omDlex  social 

2d^  -teeJkS-  to*s2  !KI  HnJr  f**  ' ^ constructed 

ony  uien  Qieaiced  to  see  wnat  kinds  of  responses  are  produced. 

to  modelers  have  turned  Increasingly 

In  "odels  of  co^lex  systems  that  are  progranable  on  digital  computed 

tH  cuwnt  siwilatlon  efforts  depend  heavily  upon  the  use  of  a conuter 

that^w  te  ^leldy  mass  of  data  and  InterrelattelSips 

^T*  considered.  Ue  should  stress  that  using  a ccanuter  does  not  oer  'U 

Ti 

’'•  ufr^hLwI!  «r*4llr  5|99er  and  faster  computers  has  helped  the  wdeler  to  manipulate 

rH  F*  w"*'" 

^ analyze  and  predict  the  behavior  of  coaplex  physical  systos  (such 

and^t.o^thl"Mntc^/V^4  control)  by  Inspection,  thinking  a lot.  debate,  coiproiirfse. 

Intuition.  Instead,  engineers  turn  to  laboratory  prototypes  and 

to  ®“‘  '•'y  expenslvj^krt  is  WW. 

leasrar^r^of  Involving  systems  that  are  at 

iilif  MllS^Ldii^t^^^lI'^h!!®'^  complex  than  purely  physical  systems  and  to  make 
Tii  judgnents  by  the  more  accustomed  art  of  debate,  conrrtxaise.  and  so  on 

the  utlin^lEEl  E!S.ir.«2*Sr'  ’’1”  W)’!'*'  »'!  ptctorlel  aodels  are  stailar’to 
suhstit^>«-\lf®*’*i  computer  simulations  In  that  they  provide  explicit 

they  are  static  Most*of*t^T  t Yet  these  models  represent  physical  subsystems  and 
y are  static.  Most  of  the  Interesting  (and  least  understood)  xen^rte  nf  a^mnnn 

wlarconM^t*!?*  r°"*  DED?  *!!*  Interactions  over  time  of  Its  physlcafand 

S aiySr^l  n«w.  «>“  scale  models  canno?  hi  S 1pS"J 

- ?sF’ 

b*£Ci5“ht3“R  "-t"''- „ 

are%S?ll  Ind  III  to  systems.  While  our  comments  In  this  paper 

of  11^10  applied  to  any  weapon  process  procurement  system  or  any  phase 

Dlanninl'lh.fl  Ir  ^ control),  our  particular  example  In  this  paper  focuses  on  the 
Jf  r"?®"  procurement.  We  therefore  focus  our  disluislon  ol  lie 
app^ch.  that  of  system  dynamics,  to  building  computer  simulation  models  of  that 
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Systcw  Dynamics 


Systew  dynamics  was  developed  during  the  1950*s  by  Jay  U.  Forrester  and  his 
colleagues  at  the  Massachusetts  Institute  of  Technology  to  help  in  their  study  of 
industrial  systems  (Forrester  1961).  Over  the  past  decade,  the  technique  has  been 
widely  applied  to  simulate  a variety  of  social  systems:  the  world  copper  market 
(Ballmer,  1960),  natural  resource  use  (Behrens,  1973),  planning  and  control  for 
cowunity  hospitals  (Nasson,  Moody,  and  Stubbs*  1972),  decay  of  urban  systems 

^ energy  system  (Naill,  1975),  regulation  of  electric  utilities 
(Ford,  1976),  and  logistics  systems  (Stanhagen  and  Gardiner,  1976).  The  technique  Is 
most  widely  known  for  its  use  im  the  study  of  The  Limits  to  Growth  in  Dooulation  and 
industrial  activity  In  a finite  world  (Meadows  et  a1.,  1972).  ^ > 


The  heai^  of  the  system  dynamics  paradigm  Is  the  view  that  social  systems  belong 
to  a general  class  of  nonlinear,  feedback  systems.  To  facilitate  the  representation 
of  Mch  systems,  the  system  dynamics  technique  allows  for  easy  representation  of 
nonlinear  relationships  and  chains  of  relationships  that  close  on  themselves  to  form. 
TMdback  loops.  It  Is  In  the  interlocking  structure  of  multiple  feedback  loops  that 
tte  system  dynamics  practitioner  seeks  to  find  the  explanation  of  the  dynmalc  behavior 
of  complex  social  systems:  . 


It  Is  In  the  positive  feedback  form  of  system  structure  that 
one  finds  the  forces  of  growth.  It  is  the  negative  feedback, 
or  goal-seeking,  structure  of  systems  that  one  finds  the 
causes  of  fluctuation  and  Instability. 

(Forrester,  1968) 


To  build  a ^stem  dynamics  model,  one  follows  a series  of  steps  described  In  the 
preface  to  the  Dynamics  of  Grcnrth  In  a Finite  World  (Meadows,  1974)  or  those  discussed 
in  ^e  Introduction  to  Urban  Dynamics  (Alfeld  and  Graham.  1976).  Generally,  the 
following  steps  are  required: 


State  the  Probl^.  At  the  very  beginning,  the  modeler  should  specify  the 
issues  the  model  Is  to  address.  Usually  this  Involves  a description  of  a 
particular  system  that  Is  behaving  "poorly-  and  a question  of  which  policy 
proposals  should  be  adopted  to  make  the  system  behave  -better"  In  the  future. 
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^^P  Dete^ne  the  Key  Factors  and  Their  Causal  Interrelationships.  In  this  step, 
the  modeler  lists  the  key  factors  that  are  believed  to  cause  the  behavior  of 
concern.  The  Interrelationships  among  these  factors  are  Identified  and 
portrayed  In  the  form  of  causal  diagrams.  An  example  of  a causal  diagram 
in  the  weapon  procurement  system  is  given  In  Figure  1.  As  the  niaerous 
Interrelationships  In  any  system  are  Identified,  some  series  of  relationships 
form  a closed  chain  or  feedback  loop.  In  Figure  1,  such  a loop  Is  noted  by 
aXp  . It  Is  in  the  interaction  of  these  loops,  that  the  modelers  seek  to 
explain  the  behavior  of  the  system  under  study. 


5^®P  3.  Formulate  Model  In  FI  ^Diagram.  In  this  step  the  representation  of  the 
system  is  reformulate  in  the  form  of  a flow  diagram  which  facilitates  the 
^entual  representation  In  the  form  of  a set  of  difference  equations.  The 
flw  diagram  Is  especially  helpful  In  distinguishing  between  flows  of  material 
and  flows  of  Information.  Moreover,  the  flow  diagram  forces  the  modeler  to 
Indicate  explicitly  any  delays  or  nonlinear  relationships  included  in  the 
model. 
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Step  5. 


The  use  of  causal  loop  diagraas  and  flow  dfagraas  not  only  helps  the  aodeler 
to  keep  track  of  his  progress,  but  It  helps  coominlcate  the  luportant 

****  potential  users  as  well.  The  use  of  aesilngful 

naws— n«s  that  can  be  seen,  "felt*,  or  talked  about— together  with 
the  dlagran^  aids,  gives  properly  constructed  systea  dynaalcs  aodels  a good 
cha^e  of  bridging  the  communication  barrier  between  the  aodel  builder  and 
model  user.  This  barrier  has  been  described  by  Goldie  (1970)  as  follows: 


When  we  attempt  to  use  the  new  techniques  that  manageaent 
advocate,  we  suddenly  find  that  we  are  out  of 


the  1^.  A bright  young  nan  takes  By'pr^1en'’may'and 
translates  It  from  managerese  to  computerese. 


Step  4. 


In  speaking  about  this  opportunity  for  bridging  the  conainication  barrier. 
jJ  ***  systea  dynaalcs  approach  as  ‘common  sense 

difficult”**  elaborate  techniques  are  described  as  ‘coommi  sense 

J I , r - ♦ 1 f -fw  • , - 


^ P«''**ter  values  aust 

tV**'’"**^'  Each  of  the  paraaetm  of  a systea  dynaalcs  aodel  Is  estlaated 
Mlvldually  fra  the  best  Inforaatlon  available.  This  Information  can  take 
acewlie  physical  aeasurements  and  make  use  of  voliaes  of 
ifln  ^***/**  ®"«  0^  T*»  spectnai)  or  exert  opinion  and 

i^Wldual  Intuition  (at  the  other).  The  use  of  expert  opinion  In  the 
aosenra  of  more  formal  data  to  help  In  parameter  Identification  tends  to  make 
^ data  purlsu  squirm;  yet  to  rule  out  subjective  Inputs  If  they  are  the 
only  source  of  Information  available  Ignores  the  fact  that  aost  of  the 

system  Is  contained  In  the  heads  of  people 
recorted  In  the  form  of  time  series  or  cross  sectional 

shtei  ®r  ■“*’^"9  techniques  prefer  to  Ignore  relatlon- 

shlrs  for  which  hard  statistical  data  is  not  available.  Leaving  such  rela- 

r o «S’9n1n9  them  the  parameter  value  that 

wrong:  aero.  By  Incorporating  subjective  judg- 
StiLln-  rhl  »alu«s,  the  system  dynamics  practitioner  can 

sensitivity  testing  those  areas  where  further  debate  and 

* "wdel’s  behavior  Is  generally  Insensitive 
» Pai'aieter  values,  scarce  resources  should  not  be  wasted  In  collect- 
ing  data  to  estinate  them. 


Ggnerate  Initial  Output  and  Increase  Confidence  In  the  Model.  Once  parameter 
re  lues  have  been  esUmaled,  OyHAHO  equations  of  a systeTdyniil^ 

software  package  usually  used  to  con- 
Jhl  dynamics  wodels.)  If  the  Initial  output  generated  by 

returef^to^slln  systea* s behavior  well  enough  the  modeler 

’"*«'"act1ve  process.  Me  should  emphasize 
«1^;.  dynamics  models  do  not  automatically  reproduce  historical  trends 

estimated  Individually.  This  procedure  Is  quite 
approaches  tdierein  m^el  parameters  are  estimated 
of  liili  historical  trends  over  a certain  period 

bLwiIh--  I . ’ " "®  surprise  that  such  aodels  are  capable  of 

PTOvtoing  extremely  close  fits  to  historical  treiKls  over  the  time  period  used 
In  the  parameter  estimation.  They  are  designed  to: 


Generating  a reasonable  pattern  of  behavior  - both  over  soae  historical 
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period  and  into  the  future  - is  but  one  of  several  tests  that  should  be 
perforaed  to  increase  confidence  in  a nodel.  Sensitivity  testing  should  be 
perfonwd  to  ensure  that  the  model  does  not  respond  unrealistically  to  changes 
, . in  particular  input  valMS.  The  response  of  a model  to  disturbances  in 
. . exogenous  inputs  should  be  examined  for  plausibility.  These  and  other  con- 
fidence tests  are  described  in  Forrester’s  Industrial  Dynamics  (Forrester, 

' ; 1961). 

Step  6.  Siwiate  tfci^  Efflirts  of  Proposed  O— »8.  In  this  final  step,  the  modeler 
simulates  tat  anVlcts  of  pending  or  acttml  changes  to  determine  the  effect 
on  the  system.  ^ 4 

There  are  at  least  two  kinds  of  changes  that  can  effect  system  behavior.  First, 
there  are  policy  changes  that  are  under  the  control  of  the  SPO.  For  exam;:1e,  an  SPO 
may  be  considering  the  merits  of  giving  a go-ahead  on  stretching  the  production 
schedule  vis-a-vis  simply  slipping  the  entire  schedule  by  a uniform  aowunt.  A second 
kind  of  change  is  one  imposed  on  the  system  from  the  outside.  For  example,  a wage 
Increase  for  production  workers,  a rise  in  the  cost  of  material  used  in  construction, 
or  congressional  changes  in  the  authorized  budget  for  the  program.  In  these  instances 
there  are  two  questions  to  be  asked:  (1)  what  is  the  effect  of  the  change  on  the 
system  if  I (the  SPO)  do  nothing?  and  (2)  what  action  can  I take  as  the  sjrstem  ranager 
in  the  face  of  this  change  to  improve  system  response  to  it?  In  Step  6 each  kinc  of 
change,  as  well  as  combinations  of  changes,  are  examined.  . .<• 

A Widget  System  Acquisition  r , hr* 

Ue  chosen  to  Illustrate  our  approach  with  the  acquisition  process  surrounding 
a piece  of  well-known  ailitary  hardware;  the  widget.  In  this  instance  widgets  a^ 
"luRps”  of  material  that  normally  weigh  about  one  pound  each.  Ve  have  chosen  this 
very  simple  system  to  illustrate  our  approach  so  that  the  approach  can  be  focused  on, 
not  the  details  of  a very  complicated  weapon  system.  Figure  2 shows  the  various  kinds 
of  changes  that  could  be  c iaroined  in  any  acquisition  process  and* represents  in  tree 
diagram  form  tne  shopping  list  of  potential  questions  that  might  be  asked.  To  illus- 
trate our  approach  and  keep  the  task  (and  presentation)  manageable,  we  have  further 
focused  on  the  budget  changes  branch  in  constructing  our  system  dynamics  simulation 
model.  Our  model,  in  its  current  version,  does  have  limited  capability  to  explore 
other  kinds  of  changes  but  could  easily  be  expanded  to  Include  the  system  structure 
capable  of  answering  the  questions  suggested  by  the  other  branches  in  Figure  2. 

Moreover,  it  would  have  been  possible  alternatively  to  focus  on  system  performance, 
schedule,  unit  performance  (performance  per  copy)  or  even  all  four.  Me  should  note 
at  this  point  that  the  tree  diagram  in  Figure  2 (as  well  as  our  model)  strongly 
suggest  that  there  are  many  combinations  of  changes  that  are  not  possible.  For 
example,  a coirmonly  desired  set  of  changes  that  face  many  SPO's  is  for  more  performance 
per  copy,  shorter  schedule  of  production,  and  lower  costs  per  copy  that  lead  to  lower 
system  costs  (total  buy),  more  copies  in  the  system,  higher  system  performance,  and 
early  delivery!* 


♦See  Figure  3 for  the  c;:usal  1-op  diagram  that  describes  the  total  system. 
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'figure  2.  Tree  diagram  of  possible  acquisition  system  changes. 


• perfomumce  per_ 
copy  changes 


There  is  also  an  Inportant  technical  point  about  our  model.  The  printout  over 
♦'•e  is  artificially  linked  to  the  time  dimension.  What  this  means  is  that  in  a larger 
' Sion  of  the  model,  the  calculations  that  now  appear  over  time  would  probably  all  be 
Done  in  one  solution  interval  increment  (i.e.,  one  unit  of  time).  However,  so  that 
w“  could  watch  the  model  hunt  around  for  the  correct  response  to  changes  we  intro- 
( ed,  we  have  a model  that  corresponds  more  closely  to  a very  patient  SPO  with  a 
•.•lid  calculator  making  a series  of  hand  calculations  over  and  over  again  converging 
on  the  final  and  new  "steadly  state*  of  the  system  in  response  to  changes  introduced. 

‘ In  a larger  and  more  enhanced  version,  this  “watching  a model  converge*  is  of 
little  interest  other  than  to  get  a feel  for  how  the  model  functions,  so  the  entire 
nndel  presented  here  would  be  a component  of  a larger,  more  realistic  and  time  based 
. tl. 
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Figure  3*  Causal  Loop  Diagram  for  Basic  cost, 
SOieduling  and  performance  Xnteraotioni 
Systems  Acquisition 
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It  was  assumed  In  constructing  our  model  that  the  widget  program  Included  an 
ROT&E  phase  - totally  co^ileted  for  all  practical  purposes  - and  a production  phase, 
not  yet  started.  Usiig  the  model  as  a planning  tool,  we  monitor  budget  discrepancy 
(program  budget  co^tared  to  estimated  program  costs,  which  Includes  ' ^Ixed  ROT&E 
(amount),  niaber  of  hours  discrepancy  (total  straight  time  hours  remired  versus  total 
straight  time  hours  available),  and  system  performance  discrepancy  (compared  system 
performance  goals  with  actual  system  performance).  .Mso  Included  Is  a copies  change 
rate,  performance  oer  copy  change  rate,  actual  system  performance  change  rate  and 
madier  of  daiys  for  schedule  change  rate.  ..x  _ 

There  are  no  system  delays  Included  In  the  model  In  this  first  configuration. 
Hwever,  delays  (allying  for  the  OSARC  decision  process,  for  example)  could  easily 
be  added.  • Figures  4,  S and  6 portray  the  schedule  (HDISC),  cost  (BOISC)  and  per- 
fonoance  (SPE^)  subniodels;  Figure  7 Is  the  total  model,  ^pendlx  A defines  all 
terms.  Including  fixed  values  and  table  functions.  Appendix  B is  the  program  state- 


ments. 


Assimgitl  ais  and  basis  fixed  values  were  set  for  a system  that  Is  scheduled  to 
produce  paper  weights.’  Assimgitlons  are  as  follows:  i ' - 

. . 

- number  of  workers  Is  constant  l • j . ' ' ' 


If  additional  production  hours  are  required,  the  number  of  days  will  be 
Increased  / i . 


- overtime  will  not  be  used 


J 


■ ^ 


^ basic  ”days*  calculation  is  driven  by  number  of  production  copies  and 
performance  per  copy 


U 


- each  paper  Height  neighs  1 pound  (which  is  the  desired  performance/copy) 


h ' ‘ ■ 


'/  i 


\ . 


Following  design  and  check-out  of  th^  DYNAMO  model , It  can  be  expanded  to  fit 
more  conplicated  products  such  as  aircraft  or  missiles. 


♦In  an  enhanced  version  any  or  all  of  these  assumptions  could  be  relaxed  as 
necessary  to  reflect  the  real  system. 
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- the  toUl  production  run  Is  1000  copies 

• production  schedule  Is  25  days  (5  copies  per  hour) 

• estimated  cost  per  copy  Is  $10  ...  J 

* • \ ■ fi  - 

--  program  budget  Is  $10,000 

- .1 , ' 

Model  Print-outs;  Verification  Checks  f [ 
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The  Initial  model  ran  when  first  Inserted  Into  the  computer.  The  pr1nt*out  of 

the  first  run  Is  Figure  a. 

^ 

It  was  then  necessary  to  examine  the  model's  validity  by  Introducing  a change 
and  then  removing  the  change  and  observing  If  the  responses  to  the  change  made. sense. 
The  modeU  given  this  kind  of  a change,  should  respond  and  then  should  return  to  the 
oiiqlnal  steady  state.  Figure  9 shows  the  effect  of  Increasing  the  program  budget 
by  $2000  at  t*10,  folloMd  by  a decrease  of  $2000  at  t»20.  sim  confidence  Is  gained 
In  the  model  as  we  see  the  data  (cop1es*C,  N days«M,  EPOC-R  and  PROBUO-*/  Increase  . 
appropriately  at  t»10  and  then  return  to  normal  after  the  decrease  at  t*20. 


Additionally^  to  confirm  that  the  solution  was  not  an  artifact  of  the  solution 
Interval  used  In  the  s1«ilat1on,  the  solution  Interval  of  1 was  dropped  to  .03125.  The 
pr1nt*out  In  Figure  10  showed  that  the  change  affected  transients  but  had  no  Impact  on 
the  steady  state  solution. 

' ^ I - ^ 

Model  Pr1nt«outs  * Charging  Inputs  / ' ' -.V 

At  this  stage  In  the  simulation  design,  there  Is  no  capability  to  input  a change 
to  system  performance  goals,  program  schedule  or  performance  per  copy.®  The  model 
only  permits  showing  the  inpact  of  a program  budget  change  or  of  chanqinq  the  i 
following  variables:  , ' 

1)  PR06UD  (Program  Budget) 

2)  LABH  (estimate  of  Labor  Hours  per  Copy  as  a function  of  Performance 
per  Copy) 
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She  capability  to  change  these  factors  can  be  provided  In  a later  version 
of  the  simulation. 
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3)  STHPWD  (niaber  of  Straight  Time  Hours  Per  Uorker/Day) 


\ 


e*  4)  TINC  (Total  Indirect  Cost  of  Manufacturer)  t 


6)  STHR  (Straight  Time  Hourly  Rate) 


5)  ROT&E  (ROTtf  Costs) 


C D ^ O 


X 


7)  RMPCN  (Anount  of  Material  Per  Copy  Normal) 


T 


8)  GOAL  (System  Performance  Goals) 


* ^ ,9)  COHN  (Cost  of  Material  Multiplier) 

Mile  all  of  the  above  nine  variables  can  be  varied  and/or  fixed 


of  f Xing  2»  3 or  4 of  the  prime  variables  (days,  performance/copy,  budget  and 
number  of  copies)  may  not  be  possible.  If  data  provided  for  Input  Is  not  correct, 

the  model  in  11  Indicate  whether  that  data  Is  critical  and  whether  it  mtst  ^ 

determined.  At  the  sane  time,  the  model  will  demonstrate  which  data  need 
reasonably  accurate  - perhaps  could  even  be  only  loosely  determined  or  estinaceo  - 
since  the  four  prime  variables  may  be  shown  not  to  be  sensitive  to  large  variations 
in  those  data.  A very  useful  capability  may  be  the  opportunity  to  request  proposals 
from  the  contractors  and  then  to  verify  the  validity  of  their  responses.  Should  a 
contractor  state  that  requested  schedules,  budgets,  etc.,  cannot  be  met,  the  model 
prwides  a check  for  what  may  be  an  aggregation  problem  rather  than  an  Impossible 


demand. 


^ To  test  the  above,  a very  simple  analysis,  not  unrealistic  In  these  days  of 
climbing  wages,  was  made.  Wages  were  increased  10%  across  the  board.  As  would  be  - 1 

Mpect^,  the  labor  hours  per  copy  increased  ($9.00  up  to  $9.73),  nuatoer  of  copies  to  ; ) 

be  built  with  a fixed  program  budget  dropped  (1000  copies  down  to  924.8).  and  system 
^rformance  discrepancy  changed  (from  0 to  75.186).  This  print-out  is  shown  as 
Figure  11.^  - i 

^ - { ) ' 

Applications  to  Weapon  Systems  r.  - ^ — \ 

The  next  conment  from  a persistent  astute  questioner  might  be: 


“Alright,  I understand  generally  what  you've  been  describing. 

But  it  seems  to  me  that  paper  i^lghts  are  a far  cry  from  F-15*s, 
Advanced  Attack  Helicopters  and  Aegis  missiles.  Can  that  model 
be  used  with  production  Items  incorporating  thousands  of  moving 
parts,  millions  of  ' chips, ' and  exotic  alloys?" 


Three  additional  results  obtained  by  varying  Inputs  are  Included 
Appendices  C,  0,  and  E. 
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• The  answr  would  have  to  be,  "No,  model  could  not  be  applied  to  one  of  today’s 

•*  «*<»"«*«<«.  «ction  by  section, 
the  expenditure  of  a relatively  small  amount  of  dollars  - it 
would  provide  a reasonably  realistic  representation,  sufficiently  accurate  and 
reliable  to  provide  useful  information  to  harried  decision'makers.” 

let’s  study  the  performance  section  (Figure  6)  of  the  model,  for  exMle  and  a«k~ 
il!**^**^^!!!^?!**®**^**  **  required  to  start  the  transition  from  modeling  thepaperweiqht 
^<5?  acquisition  process.  First,  the  system  performanc^ooal^  ' 

DrSd^i^tT**S**  simple,  one>parameter  requirement  that  the  progrma’s 

production  items  weigh  a total  of  1000  pounds  (each  pap^ight  weiohs  m o^l 
Instea^the  system  goal  now  might  be  tTdestriy  in  tS^air  U peJSt  ofl  KSJkino 
enemy  boeters  by  firing  a toUl  of  T missileT^  a n^r  of^rf^l  attacking 

maior“^**a!^n?Id®'^“i4'^®?‘  (PIWBUO)  managers  are  alerted  to  the  likelihood  of  a 
UfhfLi®**  1?  earlier  in  the  pape*',  that  actually  happened  to  the  F-15  (and 
others)  earlier  this  year.  The  proposed  decreased  budget  amount  is  picked  uo  in  the 
m^l  as  a budget  discrepancy  (BOISC)  between  PROBUO  and  EPROC;  that^result  feeds  into 
This  decreasing  the  nuaber  of  F-15’s  which  can  be  purSased? 

This  SKtim  of  the  model  now  goes  through  an  iterative  process . adjusting  the  nimd>er 

;JcSS{“o5erti2"hJL'"M“f^“f  dis^oSu  ;re"tSS»"SS" 

acc^t,  overtime  hours  (if  previously  scheduled)  are  reduced  and  so  on  - When  the 
^r  of  copies  being  procured  stabilizes,  that  figureis  fed  S the^ctualJystL 

KJfpTSSfSlilinr?^  n„^S2-r  “""t.  the  perfon5a“ce 

and  the  WDM  system  performance  change  rate  (SPERFC). 

(SPERFA  or  actual  system  performance)  of  the  two  - cop^s  and  rer- 
compared  to  the  system  performance  goal  (GOAL).  As  expected. 

IS  cw  proxies  . duc^wey  ).  .yst- 

fbllwill^-’  decision  makers  can  see  a computer  generated  break-out  of  the 


- liunber  of  Copies  Being  Procured  (with  reduced  budget) 

- System  Performance  Discrepancy  (between  goal  and  now  estimated) 
Nunbei^of  Days  Scheduled  (to  produce  the  reduced  quantity) 

- Indirect  Costs  Per  Copy 

acqu^r2)^*^  (change  due  to  lower  quantities  being 

- Labor  Costs  Per  Copy 

- Manufacturers  Cost  Per  Copy 

a 

If  the  lowered  capability  to  destroy  percent  of  eneew  bond>ers  is  unacceotablo  t-* 
aireMft"*"??!!!^^^^^  propose  increasing  the  nunber  of  missiles  to  be  cJ^Hed’  pet 

increases  as  does  labor  houre 
♦ K T ^ changes  is  an  increase  in  actual  system 

higher  performance/copy  times  the  reduced  quantity  of  aiixraftl 


COPIES 

SPERFO 

NOAYS 

I.NCPC 

NATCPC 
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■Isslles*  would  cost  a specified  extra  tuinber  of  dollars.  The  ultimate  decision- 
maker then  has  a choice  between  the  Initial  reduced  quantity  and  reduced  system 
performance  and  reduced  budget  versus  the  same  reduc^  quantity,  less  of  a reduction 
In  system  performvice  and  a bud9etaf7  position  between  the  Initial  and  first  reduction. 

Of  course,  the  model  would  allow  consideration  of  a program  stretch-ouc,  without  * 
reducing  total  quantity  to  be  produced.  Here,  however,  the  indirect  costs  would 
change  and  that  result  would  have  to  be  evaluated.  , ^ ^ 

. Other  sorts  of  questions  that  might  be  asked  of  the  simulation,  so  to  speak,  could 
Include  adding  performance  per  copy  parameters  such  as  take-off  weight,  turn  radius, 
time  to  altitude,  maximum  altitude,  and  so  forth.  Each  of  these  can  be  tied  to  a 
discrepancy  Indicator  as  well  as  p^ormmice  change  rate  values.  It  would,  therefore, 
be  possible  to  assess  the  Impact  of  Increased  weight  on  each  of  these  and  to  also 
design  In  trade-off  decisions. 

Other  methods  for  handling  system  performance  analysis  might  Include  selecting 
five  or  six  major  Indicators  of  performance  and  assigning  percentages  to  the  current 
estimate  for  each  as  conpared  to  the  goal.  Again,  discrepancies  could  be  calculated 
for  each  or  system  performance  could  be  a number  based  on  the  product  of  copies  to 
be  procured  times  the  product  of  the  six  major  Indicators  averaged.  These  methods 
mights  be  considered  similar  to  that  used  with  the  C-5A  when  Initial  program  goals 
were  broached  In  terms  of  each  C-SA  being  equivalent  to  a certain  number  of  C-14rs. 

The  question  of  where  the  current  developmental  model  stood  v1s-a*v1s  the  goal,'  was 
answered  by  saying,  "This  model  would  be  the  equivalent  of  2.5  C-141's." 

Conclusion 

As  we  admitted  earlier  - and  It  Is  worth  repeating  again  here  • our  m^del  Is  very 
primitive  In  Its  present  form:  It  was  designed  to  be.  Me  do  not  recommend  Its  use, 
and  there  are  perhaps  other  methodologies  'that  can  deliver  the  same  results.  Mhat 
we  do  suggest  Is  that  this  approach  leads  to  a simulation  model  that  Is  easy  to  con- 
struct and  use  by  an  SPO  who  Is  Interested  In  managing  a system,  not  becoming  a com- 
puter or  modeling  expert.  This  primitive  model  can  easily  be  expanded  as  necessary 
to  reflect  the  real  system  within  which  the  SPO  finds  himself  making  decisions  and 
expanded  In  terms  of  his  own  inana^rese,  not  computerese.  Anything  he  considers 
Important,  If  he  can  describe  It  In  ordinary  English,  can  be  Included.  By  developing 
a slaple  simulation  model  and  showing  how  It  proceeds  through  the  calculations 
necessary  In  a few  example  changes,  we  hope  we  have  Interested  you  in  a way  of  tackling 
much  more  complicated  systems.  We  hope  we  have  Interested  you  In  a new  - and  poten- 
tially helpful  - management  tool  for  understanding  and  managing  cost,  performance 
and  schedule  changes  Interactions  In  the  Weapon  System  Acquisition  planning  and  control 
process. 
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Potmtfally  there  might  be  an  even  greater  benefit  through  using  the  DYNAMO 
simulation  as  a research  tool  to  investigate  poorly  understood  phases  of  the 
«j.  sltlon  p^ss.  So.  tf.no  .»•<  cl.1.  iittf.^rrofSS)2ncruIS"S 
to  furnish  e realistic  learning  tool,  me  do  not  claim  fOr  a 
semd  that  t^l  fwiliarity  in  its  use  and  workings  will  eliminate  cost  growth  ? 

•"?  s^ule  slips.  He  do  insist,  however,  that  usITftte 
expand  mo^l  would  permit  decision-makers  to  assess  in  advance  the  results  of 
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APPENDIX  A.  A DEFINITIOff  OF  TER>^ 


OOOlOprobud  ' ^rodraa  budd«t 
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00040 rd to 
O0050«ci>c 


ostioated  prodraa  costs 
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00080co>idS  ^itUBb«>nor~l:dfaes~befiU'^rdba^^ 
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OOllOaapcn  aaiount'~bT~ aaterAaT~psy~gina>v  horaal  ~ 
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OOrtOcoMPu  cdst~of~matepxar>df~cd«i — 
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ooiTorabtB — ^tbcar-dostr-or-rabdi* — — 

OOlSOtstrtc  total  streisht  time  costs 
OOiyOsthr  straisht  time  hourlu  wade  rate 
00200tsth  rs  to  ta  t-numbe  r-ofu  t r a iaht-tl  me-hours- 
'00210sthpwd  .number  of  hours  per  worker, per  daw 
00220ndaws  number  of  daws  scheduled 


00230ndcr,  number  of  daws . for.  schedule  chande 
®®240hdlsc  oT  hours  discrepancy 

00250tsthrT— total-stratdht  time  hours^  reoui  red 

002601hpc  . labor  hours  per  copw 
002701abh  labor  hours  table  function 

00280perf pc  Fe  Ff drmance~per~copw 
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00300sperfa  actual  swstem  performance 
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TWE  SflUKB  or  Dismmoii  to  phdject  cost  awd 
E.B.  Cbchran,  A.J.  Rowe 


I.'  mrnooucnoii 


r < 1 


There  has  been  auch  attention  to  the  aesslve  ovemns  of  cost  and  delivery 
prxwlses  In  U.S.  allltary  progrSB.  The  e^rhasls  on  corrective  action  has  focused  on 
developing  better  —lageaent  techniques.  This,  however,  overlooks  the  aajor 
contribution  to  overrun  created  by  the  concurrency  of  product  developaent  with 
production.  In  the  presence  of  an  urgent  coreritaent  to  deliver  operable  wits. 

‘ Concurrency  of  developnent  and  production  affects  production  through  the  In- 
evitable design  delays  and  changes.  The  resulting  disrwtion  of  operations  Is  the 
Hjor  reason  for  project  cost  overrws  (often  rumlng  fifty  percent  or  aore)  and 
schedule  slippage.  The  general  consequences  of  concurrency  have  long  been  recog- 
nized. But  the  causal  n^anlsas  have  not  been  fully  wderstood.  and  so  the  possi- 
bilities of  avoiding  or  altigating  the  problea  have  not  been  exploited. 

The  research  reported  here  provides  an  expanded  wderstanding  of  the  disruption 
process  and  proposes  specific  ways  to  deal  with  It.  An  Inportant  conclusion  Is 
that  disruptions  of  both  coonerclal  and  ailltary  prograas  are  created  by  essmtially 
the  same  causes,  and  so  are  responsive  to  slailar  preventive  actions. 


. , , - Prior  Research  on  Disruption  Effects  - • ‘ 

Early  work  on  the  S-Curve  by  Cochran(l)  Shows  that  aajor  overruns  In  direct 
labor  were  due  to  tine  ccopresslon  which  resulted  fron  Inadequate  time  fbr  design 
and  production  startup.  This  was  further  developed  by  demonstrating  nunerous 
ways  In  which  such  costs  are  caused  by  deslw  delays  and  changes,  production  In- 
terruptions and  related  manpower  turbulenceV^) . The  S-Curve  can  also  be  used  to 
estimate  startiq)  costs,  which  are  thenselves  a less  serious  fora  of  disruption  to 
regular  production  operations.  Later  development  of  a computer  model  further 
demonstrated  the  effect  of  design  delays  and  changes  on  complex  steel  fabrication 
work.  Detailed  diagrams  fbr  the  labor  disruption  process  then  described  explicitly 
how  design  wcertainty  causes  disruption  In  production  runs.  This  work  was  supple- 
mented by  development  of  a conputer  nodel  to  estiaate  the  effect  of  concurrency 
events  on  labor  hours. 

Sines  the  late  1950's.  the  Rand  Corporation  has  conducted  studies  to  describe 
the  Inherent  uncertalntly  of  the  product  developaent  process.  From  early  1960  to 
date  It  has  Issued  a nunber  of  Increasingly  specific  reports  to  show  that  design 
and  production  concurrency  accompanies  large  overruns.  Wileh  were  avoided  when 
development  work  substantially  preceded  production.  In  this  work.  Rand  developed 
quantitative  measures  of  the  degree  of  advance  In  state  of  the  art  (SOAA)  for  spe- 
cific products,  and  demonstrated  Impressive  correlation  between  that  degree  and 
the  proportion  of  cost  overrw  fbr  cases  of  desigi  and  production  concurrency.  As 
a result,  Rand  researchers  reconnend  reducing  uncertainty  In  advanced  technology 
programs  by  return  to  "Incremental*  product  development**  - a procedure  which  ha' 
long  been  used  by  manufacturers  of  commercial  products  with  stable  design. 


Not«:  * Copyr1ght®1977,  E.  B.  Cochran 

**  See  the  selected  readings  list  at  the  end  of  th1«  paper,  especially 

Items  2.  4,  8.  9.  11.  17.  19  and  21. 
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• Farther  Eesearch  - 


![***'*f*  *“”•  Jointly  by  cbc  Oalwmlty  of  Soutben 

“*  «««orlb3  .bove.  Tbl. 

. b«»  JJX^jLTS'JS^JLr""-  “•«»“  .m  ».  P*11.1«I  to 

Of  ,J**.?f**  “j®'  project,  la  detail,  yla.  briefer  reeleo 

dL^TL  of  dlemptloa  were  Identified,  nhleh  wre  t»- 

“te  o-o^ol  1«  Judging  the  propensity  for  dlm^tlon  ^ 

‘.  ■’  o De^ee  of  tecbnologlcnl  uncertainty,  based  on  sdvMcc  In  atate  ' 

interrelatedness.  ^ ,^v»  -v.^ 

^ • • 

. * . ••  refl«:ted  In  the  ti«  co^teaalon  of  deUuety 

*•“*  to  overlap  of  design  and  production  phases  ^ 

11  * *•  *•  *-  ■ * 

. * lavel  of  resources  applied. 


I-'-. 

.»fi  itof.t'aslijr 


’•  * ^L!LS!rji.Sr°^  sorted.  «|  by  -rket  requlrenents  and 

“’.J"**^*’*  «»eot«  occur,  and  that  project  nanagenent  la  rea- 

«ilor«ptlon  can  be  vleued  as  a function  It  these  four 


i'  e s s 

•ore  ^^4  ***‘**^  describe  the  difficulties  of  concurrency  In 

outline  mys  by  which  to  quantify  the  cost  effect,  and  pre- 
!?  ? reco«ndatlons.  aose  attention  also  wlU  be  given  to  the 

-nage^nt  conpetence  In  a dlsrSHon  slti- 
..  concurrency  of  design  and  production.  The  current  stage  of  dlsruv- 

ssis  for  Mrked  iaprovenent  in  scheduling  procedures  over  a nuaber  of  years. 
Equally  major  Inproveneets  In  the  nanageMnt  of  disruption  are  anticipated. 


!•  Key  Concepts 


n.  BASIC  CONCEPTS  OF  DISKDPTION 


• What  Is  Disruption?  — 


"Warvpt"  aeans  to  cause  disorder  or  tnmoU.  la  bur  context  the  ten. 

relat^^**?nrf^^*^<*  ^ disorder  and  tunaoU  created  In  a progran’plan  and  In 

related  production. procedures,  which  naturaUy  are  als«d  at^alnllrJirc^^J.f" 

“Ot"  1*  the  difference  between  the  actual  cost  for  a pro- 
*?*'  "u*  "reasonably  required"  to  perfoi.  the  task  »ld 

1 k m'!  *k!  finally  delivered  to  the  customer,  on 

shall  tern  this  estlnate  the  "should-cost  as-bullt".  It  Includes 


the  estlAated  cost  of  ell  changes  incorporated  la  the  product  as  If  known  when 
production  was  planned;  It  does  not  Include  penalties  caused  by  the  late  Incor- 
poration of  changes  (retrofit » rewrk,  lost  learning,  etc.)* 

The  existence  of  a aixable  overrun  alvaps  raises  the  question  an  to  the 
adequacy  of  the  eatlMte  use!  to  detemlae  the  aixe  of  that  overrun.  For  ex- 
aaple.  the  overrun  nay  reflect  a poor  assessnent  or  errors  in  the  estlnating 
proceaa.  both  leading  to  a low  estimate;  such  a discrepancy  Itself  could  well 
be  disruptive.  Ve  shall  use  an  "objective**  definition  of  disruption^  with  the 
ahould-cost  to  be  a realistic  estimate  for  the  organisation.  Thin  auggesta 
ualng  **parametrlc"  eatlMting  siethods^  uklcb  deal  with  fundamental  character- 
latlca  of  the  product^^). 

• • f • • • * • 

The  inportanca  of  "^product  development**  in  thin  diacosalon  leads  ua  to  state 
cmr  definition,  which  foUowu  one  given  by  in  1967*: 

**Developnent  la  the  obtaining  of  a proven  caoabllitv  tm  Mnisy  a product 

or  process  that  differs  significantly  from  anythli^ebhainij  In  the  pas~ 

• ••  The  product  is  sufficiently  different  fror  pr«rvious  products  in  that 

there  are  substantial  uncertainties  surrounding  its  technical  aspects**^ 

(Enphasis  supplied). 

A prime  purpose  of  developnent  is  to  reduce  that  uncertainty  to  an  acceptable 
level.  The  term  **design'*  often  refers  to  the  output  of  an  engineer's  activity  of 
"designing**,  presumably  in  the  form  of  plans  and  specifications.  In  this  sense, 
design  is  only  a part  of  the  product  development  cycle,  which  may  require  several 
re-designs.  However,  for  simplicity  here  ve  shall  refer  to  the  entire  develop- 
ment process  as  **design**.  in  the  broader  sense  that  the  design  is  really  not  com- 
plete until  it  is  ready  for  production. 

There  are  three  basic  causes  of  disruption:  force  majeure.  concurrency  of 
product  developnent  (or  design)  and  production,  and  inadequate  planning  and  man-  - 
agement.  Force  majeure  covers  such  catastrophic  Mtside  events  as  a natural  di- 
saster. civil  disorder,  major  strike,  fire.  etc.  Concurrency  of  design  and  pro- 
duction disrupts  an  organization  by  introducing  the  uncertainty  inherent  in  the 
design  process.  This  can  have  serious  effects  on  performance  when  production 
must  proceed  before  the  design  is  complete,  in  order  to  meet  a tight  deadline. 
Obviously  the  seriousness  of  the  disruption  has  something  to  do  with  the  degree 
of  overlap  between  the  design  and  production  processes. 

Good  planning  and  management  is  of  great  importance  to  a new  program.  The 
task  must  be  well  understood  and  thoroughly  planned  at  the  onset,  adequate  re- 
sources of  men.  materials,  facilities  and  money  must  be  available  at  the  appro- 
priate times  and  the  program  must  be  administered  with  suitable  precision  and 
detailed  control  over  operations.  It  is  not  easy  to  arrive  at  a clearcut  Judg- 
ment as  to  management  inadequacy,  especially  in  the  presence  of  concurrency, 
which  can  virtually  preempt  even  the  best  management.  For  how  can  management 
understand  or  plan  tasks  in  advance  whea  the  task  may  change  materially  over  the 
course  of  the  project?  Part  IV  will  discuss  this  subject  further. 


Kote;  * Selected  readings.  Item  4.  page  4. 
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- The  laportance  of  Coocurrency  - 
It  My  give  concretenees  to  the  reality  of  4isruptioa  If  ve  look  briefly  at  a 


few  exosplee*: 


Project 

Booan  Aqueduct 
Sues  Canal 
Ponona  Canal  (U.S.) 
Indian  Bead  atonic- 
poMer  plant 
3 AEC  focUitiea 


Percent 

Overrun 

' lOOZ 
200 
70 

250 

82 


Project 

BAIT  Systen 
Voshington  Metro 
The  Great  Boatcm 
Conroir  B58 
Lockheed  CSA 
Bolin  Boyce  BB211 


\ ) 


The  ovemmn  go  as  high  os  300X,  neaning  that  the  final  project  costs  sen  «b 
to  four  tines  the  original  estimate**.  AU  these  progrons  involved  concurrency— 
to  exosple,  any  of  the  constmction-oriented  projects  involve  substantial  uoik 
beneath  the  surface  of  the  earth  vhich  inherently  involves  the  exploration  of  nev  p 
conditions^  in  addition  to  a high  degree  of  technology.  The  sharp  difference  in 
overrun  for  the  two  aircraft  coses  has  bees  shewn  to  be  related  to  a sinilor  diff- 
erence in  the  SQAA  of  the  two  projects^  while  the  Qreat  Eastern  was  a nsjor  tech- 
nological advance  in  England  in  the  1850*s* 

The  great  importance  of  coscsrrency  os  a cause  of  ovemms  is  a natter  of  di- 
rect experience  to  thousands  of  nonagers  in  high-technology  industry.  It  has  also 
been  extensively  documented  by  a series  of  Bond  Corporation  studies.  For  exMple^ 
after  some  twenty  years  of  studying  complex  development  progrons  in  the  U.S-  end 
abroad,  a 1971  report  concluded  that:# 

**High  systen  cost  and  cost  growth  appear  to  arise  primarily  from  efforts 
to  subdue  difficult  technology  on  hi^y  compressed  schedules  ...  (and  the) 
acceptance  of  optimistic  assumptions  about  the  long-term  predictability 
of  technology  and  the  cost  of  coping  with  it." 

In  suanary.  the  Rand  studies  oIj^  show  that  cost  and  schedule  probleM  are 
greater  as  the  degree  of  SOA  advance  increases,  sophisticated  management  planning 
^ control  prograu  and  incentive  contracting  have  sull  effect,  and  InproveKnt 
tltfough  better  coat  estiisating  and  Mnltoring  of  cost  growth  does  not  see.  sub- 
stantial. • Concurrency  may  not  even  reduce  developMnt  time  algnlficantly. 

In  short,  the  main  problem  lies  in  the  concurrency  itself,  with  the  uncertainty 
that  it  contains  and  the  effect  of  that  on  development  and  production  costs  in  the 
presence  of  urgent  time  schedules.  Thus,  these  studies  recossiend  incremental  proce- 
dures for  product  development.  While  the  most  exhaustive  studies  have  involved 

military  programs,  the  operation  of  concurrency  applies  equally  to  cosBerciol 
programs.  Any  industry  involving  advanced  technology  and  competitive  pressures  for 
early  delivery  will  be  a prime  candidate,  os  will  be  apparent  when  we  describe  the 
mechanisms  by  which  concurrency  produces  its  dismal  effects. 


Botes:  * These  are  generally  items  from  the  selected  readings;  most  are  from 

publications  by  Rand  personnel,  except  those  on  the  Great  Eastern, 
the  Roman  Aqueduct  and  the  two  rapid  transit  systems. 

**  In  most  coses  the  effect  of  Inflation  has  been  removed  from  these  fig- 
wtes.  os  has  the  fact  that  the  scope  of  a program  is  often  reduced 
as  overruns  develop  in  order  to  stay  within  reasonable  distance  of 
the  funding  available. 

^ Selected  readings.  39. 


< 


2.  Factom  itt  dwiciinnency  Di>ruptlon  ^ 

Exhibit  II-A  displays  the  major  factors  insolved  in  a dlsruptioii  due  to  con- 
currency of  design  and  production.  There  are  tuo  principal  causes.  Firsts  a 
high  degree  of  delivery  urgency  which  forces  product  development  to  overlap  with 
the  planning  and  initiation  or  production.  Second^  a considerable  degree  of 
technological  uncertainty  in  the  development  work  which  must  be  done. 

De livery  urgency  generally  arises  from  marketing  ccmsiderations.  The  demands 
• particular  customer » enforced  by  competitive  conditions » exerts  strong  pres- 
sure on  suppliers  to  commit  to  delivery  dates  which  arc  inherently  optimistic  or 
based  on  the  assumption  that  no  serious  problems  will  develop*  The  plan  nay  even 
be  quite  conservative  based  on  actual  performance  for  other  projects;  it  only 

a natter  of  urgency  whec  ^-ombined  with  some  degree  of  technological  un- 
certainty. Delivery  urgency  can  disruptive  even  when  the  initial  delivery 
commitment  can  be  eased*  since  much  planning  ia  done  and  work  initiated  predicated 
on  the  original  date,  k subsequent  deferral  then  creates  added  costs. 

Technological  uncertainty  is  created  by  the  need  to  accomplish  a consider- 
able degree  of  advance  in  state  of  the  art  (SOAA)*  and  intensified  by  a high 
degree  of  interrelatedness  in  the  design  and  in  the  production  process. 

The  degree  of  SQAA  obviously  has  a direct  impact  on  the  degree  of  uncertainty 
in  the  project.  High  interrelatedness  has  also*  by  making  it  difficult  to  anti- 
cipate the  ioplicatioQS  of  a design  change  through  many  areas  of  the  design  and 
production  processes.  But  perhaps  the  most  profound  impact  of  interrelatedness 
lies  in  the  sheer  work  and  time  required  to  trace  these  effects*  and  in  the  enor- 
mous possibilities  for  disruption  of  the  production  operations  underway  whenever 
a design  is  delayed  or  changes  are  amde  to  it* 


- Advance  in  SOA  - ' 

Defining  the  degree  of  advance  is  difficult*  if  for  no  other  reason  than 
because  the  technological  improvements  planned  for  or  required  by  a program  are 
often  not  understood  to  be  so  substantial  as  they  turn  out  to  be.  SOAA  is  diffi- 
cult to  quantify*  and  so  to  explain  in  concrete  terms  to  anyone  but  an  expert. 

At  the  sane  time*  the  experts  pride  themselves  in  solving  difficult  problems  and 
so  tenc  ^e  optimistic. 

It  is  becoming  possible  to  supplement  our  somewhat  intangible  sense  of  tech- 
nological gap  by  developing  concrete  definitions  of  objective  product  character- 
istics and  performance  levels*  and  Important  steps  are  being  taken  to  establish 
quantitative  measures  of  SOAA*.  However*  there  is  still  great  danger  in  starting 
a new  product  development  that  its  degree  of  SOAA  not  be  appreciated.  This  pre- 
vents proper  identification  of  the  areas  which  might  offer  difficulty*  and  so 
require  new  personnel  and  other  resources  as  well  as  suitable  alternatives.  It 
also  limits  the  establishment  of  suitable  warning  devices  that  trouble  is  brewing. 


C'' 

C; 

o 

o 

o 

o 

o 

o 

i~) 


i 


Note:  * Selected  readings*  Items  5*  8*  11*  17*  19  and  21.  For  an  early  effort 

at  rating  SOAA  see  Item  11*  page  13*  while  Item  19  represents  a I 

fairly  mature  application  of  the  technique  to  aircraft  turbine  — 

enginei:. 
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Interrelatedness  - 


I£  the  dc^ee  of  SOAK  Is  the  fundamental  driver  of  technological  uncertainty 
then  interrelatedness  is  s major  mcdtiplier  of  its  iapact  on  cost  of  development 
and  production.  Interrelatedness  of  design  boils  doim  to  the  fact  that  s change 
in  ooa  component  or  subsystem  affects  many  others*  and  of  coarse  this  process  can 
lt*Mta  further.  Interrelatedness  can  also  affect  production  and  vendor  activities* 

’ •lacs  a change  in  prodactlon  methods  or  delivery  cycle  in  one  area  or  component 
(generated*  of  coarse*  by  design  delays  and  changes)  may  affect  production  of  other 
components  or  murk  in  other  production  areas.  Again  the  process  can  become  very 
complex  with  many  ripple  effects.  In  general*  a product  operating  in  a more  advanced 
•*aa  of  technology  will  be  more  aubject  to  interrelatedness.  Aircraft  and  ahips 
pkovide  good  eza^les  of  great  interrelatedness  in'design*  but  there  are  many  others. 

Interrelatedness  in  the  production  process  has  many  c agrees  nnd  effects.  For 
****f1®*  • different  sequence  of  production  operations  may  be  required  when  several 
components  become  integral  parts  of  a major  assembly.  A |j|  sequence  for  one 

coaponent  may  require  sequence  revisions  in  several  others  to  balance  shop  activities. 
Or*  a different  process  may  be  necessary  because  new  materials  are  used  (ssy  titanium 
of  steel)  or  a design  change  requires  tolerances  which  demand  new  production 
equipment  and  techniques  of  quality  control. 

lha  interrelatedness  problem  is  probably  beat  visualised  in  terms  of  a compre- 
bensive  PERT  network.  This  network  links*  first*  all  major  components  of  the  pro* 
duct  to  show  the  cross  fertilisation  of  design  concepts  and  requirements.  It  then 
proceeds  to  the  production  process  to  show  its  intricate  sequencing  its  close 
relationships  to  design  release  dates.  Attention  to  life-cycle  costs*  of  course* 
extends  the  network  of  interrelationships  into  areas  of  operating  coats  and  field 
support  facilities. 


- Compounding  Factors  - 

TWo  major  factors  compound  the  disruptive  effect  of  sheer  concurrency:  First* 
the  level  of  resources  effectively  available  to  the  project  and  second ^ the  degree 
external  control  over  events.  The  level  of  resources  comprises  all  types  of 
resources  — technical*  managerial,  facilities,  financial*  etc.  Adequacy  of  the 
level  is  measured  by  what  might  be  termed  Organisational  slack**.  This  condition  is 
not  easy  to  define  precisely*  but  two  characteristics  seem  of  primary  importance. 

One  is  the  organization's  experience  in  the  basic  technology  involved.  This 
provides  an  invaluable  fund  of  knowledge  and  skill  in  handling  the  inevitable  unex- 
pected problems  which  arise* 'and  which  could  very  well  sweep  an  inexperienced  organ- 
ization. A second  is  the  degree  to  which  the  task  at  hand  fully  engages  all  re- 
sources reasonably  available  over  the  time-frame  of  the  project*  leaving  inadequate 
strategic  reserve  for  use  on  ixiexpected  problems.  This  can  be  a critical  flaw* 
given  the  intense  time  compression  inherent  in  concurrent  design  and  production. 

Keeping  the  project  small  in  relation  to  the  overall  organization  increases  the 
amount  of  slack  available  for  emergencies,  provided  it  does  not  encourage  assignment 
of  inadequate  managerial  talent  and  other  resources,  as  it  may  In  a large  organiza- 
tion. On  the  other  hand,  minimal  slack  limits  the  ability  to  cope  with  complex  re- 
quirements or  with  unexpected  variation,  and  increases  the  likelihood  and  the  cost 
of  disruption.  There  is  strong  analogy  with  the  queuing  approach  to  meeting  ran- 
domr  demands  by  use  of  adequate  float. 

The  degree  to  which  external  constraints  are  imposed  on  an  otherwise  normal 
process  of  product  development  can  also  intensify  disruption.  One  frequent  problem 
is  the  need  to  incorporate  design  ciianges  according  to  the  customer's  evolving 
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concept  o£  the  end  unit  and  Its  uses.  Others  Include  the  need  for  protracted 
outside  reviews,  for  ^ product  to  Interface  with  other  products  also  In  develop- 
■ent  or  to  accelerate  certain  phases  for  political  reasons. 


' t . - A VrcUalnary  Hodel  - 

Eililblc  n-B  shows  e speculative  sodel.  for' illustrative  purposes,  of  the 
IspacC  of  these  various  factors  on' cost.  Disruption  cost  Is  pictured  as  a curved 
surface.  Its  «slne  Increasing  uanoteoleslly  along  the  sees  of  technological  un- 
certainty (T)  and  deUvery  urgency  (0).  One  cost  snrfsca  la  defined  for  *vrh 
^vea  ratio  of  resources  to  tasks  (t).  Ibeae  surfaces  have  the  property  that  as 
the  resources  ratio  rises  the  cost  surface  rotates  downward.  Considerable  work 
has  been  done  to  eeasnie  the  relationship  of  disruption  cost  Co  other  factors, 
■“'•ever.  It  Is  preaature  to  propose  general  fimctlonal  chsracterlstl'~s. 

* ♦ 

2.  How  Concurrency  Develops  • • t 


. . — A Frequent  Pattem  — . *...v 

Concurrency  Is  Injected  In  uany  ways,  often  without  being  detected  In  ad- 
vance. One  all-too-coMon  scenario  nay  help  us  sense  the  reality  of  the  problen. 
Ihls  exanple  Is  not  an  acadeaic  one;  It  represents  actual  situations  which  have 
been  observed,  a nunber  of  tines  and  In  different  Industries. 

The  enterprise  eobarks  on  a progran  requiring  the  delivery  of  a specific  pro- 
duct within  a specified  tine  period.  UhUe  the  product  la  not  fully  designed.  It 
• la  soberly  Judged  to  Involve  only  existing  state-of-the-art.  fanlllar  to  the  orga- 
**^**^^®**'  Hever theless . serious  technical  problens  arise.  These  require  exten- 
sive tine  to  resolve  plus  substantial  changes  to  the  original  design  and  produc- 
tl<m  procedures,  and  schedules  and  costs  Increase  substantially. 

Intensive  e.  .erts  now  nust  be  nade  to  nlnlnlre  deviation  from  the  original 
deUvery  schedule  a:  d product  specifications.  This  further  Increases  costs,  as 
the  confusion  caused  by  change  Is  coafiounded  by  the  need  to  expend  greater  effort 
In  th«  tine  8tU.l  Available e Much  interaction  occurs  anoug  the  various  stages 
and  events  of  such  a situation  and  the  end  result  can  be  a cost  overran  of  Mon- 
strous proportions. 

A fair  share  of  the  problem  occurs  because  of  limitations  imposed  by  the  tight 
delivery  requirement.  Such  a constraint » together  with  the  delays  which  occur  and 
•the  added  tasks  which  must  be  performed,  cause  a ”time-coiiipre8sion**  effect.  This 
acts  like  a reflector  behind  a fire  to  intensify  the  effects  of  change  on  the  cost 
of  the  program.  Without  the  delivery  constraint  cost  increases  might  be  far  less, 
since  production  and  other  activities  could  be  suspended  until  development  mile- 
stones are  met. 


- Four  Steps  from  Design  to  Delivery  - 

Ebdiibit  II-C  carries  the  exanple  a step  further  by  conparing  the  original 
contract  plan  in  very  schematic  terns  with  the  results  of  a more  realistic  apprai- 
fhe  situation  and  with  the  final  disastrous  result.  The  vertical  heavy 
dotted  line  to  the  left  of  center  represents  the  actual  date  of  contract  for  the 
program,  and  the  horizontal  scale  at  the  bottom  is  calibrated  from  this  point  in 
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lacrenencs  of  five  periods  in  each  direction^  positive  to  the  right  and  negative 
to  the  left. 

Each  of  the  five  charts  shown  in  the  exhibit  contain  four  basic  stages* 
represented  by  bars  of  different  patterns.  The  first  bar  is  for  **basic  design**, 
vliich  entails  all  fundanental  decisions  on  design  and  major  production  processes 
including  necessary  prototypes  and  other  tests.  The  second  is  for  **de tailed 
design^  in  which  final  plans  and  specifications  snitable  for  use  by  a manufacturer 
are  prepared.  The  overlap* between  these  two  stages  involves  only  items  In  which 
the  work  continnlng  in  the  first  stage  will  have  no  material  effect  on  the  work 
already  oonpleted*  and  so  will  not  obsolete  work  onder  way  In  the  secemd  stage* 


o 

o 

o 

O- 


The  third  stage  is  termed  np^o^^tion  stert’-up*.  It  is  the  period  during 
which  the  organization  translates  the  plans  and  dpecifications  into  working  draw^ 
ings  suitable  for  use  by  vendors  and  shop  personnel*  develops  detailed  production 
methods  and  tooling*  prepares  detailed  plans  for  production*  establishes  its  ven- 
dor support  and  issues  necessary  purchase  orders*  plans  necMsary  facility  enhance-, 
ment  and  organization  changes*  and  then  inplcments.  its  plans  for  an  orderly  start 
of  production.  The.  fourth  stage  is  the  production  cycle  itself*  ending  with 
delivery  of  the  first  nit  for  a large  complex  product*  or  the  first  significant 
lot  of  units  for  a smeller  product. 


* The  Mechanism  of  Concurrency  - 


O- 

C) 


The  first  of  the  five  charts  shows  the  overall  contract  plan.  Mote  that  the 
design  phase  of  the  program  is  considered  to  be  essentially  conplete  at  contract 
tine*  with  only  one  time  period  remaining  for  cleanup  of  the  detailed  plans  and 
specifications.  The  third  stage*  involving  preparation  for  production*  allows 
six  tine  periods  and  can  overlap  slightly  with  preparation  of  final  detailed  plans. 
Some  overlap  with  production  is  also  permissible  if  done  with  close  attention  to 
the  detailed  precedence  relationships  inherent  in  the  production  process.  A de- 
livery date  of  thirteen  periods  from  contract  is  achieved. 


f The  second  chart  displays  a normal  industry  pattern  for  the  development  and 

Introduction  of  products  of  this  type.  While  similar  to  the  first  chart*  it  shows 
a range  of  tines  for  production  start-up  activities  since  a product  similar  to 
those  which  a company  has  been  producing  will  allow  some  reduction  in  cycles. 
lb>te  that  contract  go-ahead  is  not  set  until  final  plans  and  specifications  are 
conplete*  but  a small  degree  of  production  start-up  is  allowed  before  contract 
due  to  the  small  risk.  The  minimum  industry  delivery  cycle  may  then  be  computed 
at  sixteen  periods.  Allowing  for  the  overlap  of  start-up  with  design  and  produc- 
tion* total  flowtime  shrinks  to  thirteen  periods*  the  same  as  in  the  contract  plan. 
The  company's  contract  plan  seems  attainable*  presuming  close  similarity  of  the 
product  to  its  experience  and  do  substantive  design  problems. 

The  third  chart  shows  the  results  of  adjusting  the  industry  norm  to  what 
proved  to  be  the  actual  conditions  of  the  project.  First*  it  developed  that  the 
product  was  by  no  means  out  of  the  design  stage*  and  in  fact  had  not  completed 
the  basic  design  stage;  this  moves  the  entire  plan  five  periods  to  the  right.  The 
characteristics  of  the  product  then  turned  out  to  require  a production  startup 
cycle  of  seven  months  rather  than  six  as  planned*  and  t%ielve  months  for  production 
t rather  than  ten.  With  suitable  allowance  for  overlap*  the  adjusted  delivery  date 
becomes  period  twenty- two*  nine  periods  and  seventy  percent  later  than  contract. 

The  fourth  chart*  'Time-Compressed  Cycle"  shows  vividly  the  results  of 


vy* 
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squeezing  the  realistic  requiremeni j Into  the  contract  ti*e  span.  It  is  apparent 
that  the  contract  contained  the  seeds  of  disaster.  Basically,  the  design  tine 
canrot  be  shortened.  Hescrtheless.  produttlon  nost  begin  only  one  period  after  . 
contract  to  meet  the  delivery  date;  thus,  startup  must  begin  ioosediately.  , 

a -/ilie' Inevitable  Olsalter  - * 

; The  last  chart,  ’’Actual  Keaults”.  indicates  a typical  outcooe.  Due 'to  the 
confusion  res^ting  fron  premature  demands  for  production  start-up.  the  design  '* 
process  itself  may  even  take  longer  chan  normal,  as  indicated  by  the  dotted  ex- 
tenslon  of  the  second  stage  bar.  Startup  simply  cannot  begin  Imediately  and  also 
.extends  longer  than  normal  as  design  delays  and  changes  affect  vendors,  cooling 
and  shop  operations.  Thus,  the  company  could  not  start  production  until  four 
time  periods  after  contract,  even  later  than  the  original  contract  plar-.  - - -o 

^ < >.  , -4 

- . y"*  production  cycle  then  expands  from  twelve  to  seventeen  time  periods, 

fleet Ing  its  preaature  st^t  and  the  serious  and  repeated  disruptions  of  its 
orderly  development.  Hence  delivery  does  not  occur  uncil  period  twenty-one.  an 
extension  of  sixty  porceoc  over  the  contract  re<|uireMent . Actually*  th^s  is  a 
pretty  fair  performance  U view  of  the  fact  that  a realistic  estimate  was  for 
period  twenty-two.  The  company  performed  heroically  to  accomplish  what  it  did 
but  at  dreadful  cost* 


The  Product 


Developaemc  Proc— 


JT 


*4  er  design,  was  defined  above.  Ue  shall  now  describe  the  urocess 

Itself  in  some  detail,  as  an  aid  to  establishing  the  sources  of  cost  overrui  and 
• ‘“y  »®  *>lng,  we  shall  focus  on  "incremental”  development,  pre- 

V ously  described  as  the  aost  economical  procedur*  to  provide  a suitable  basis 
for  evaluating  costs  of  coocurrency. 

* The  Increneotal  Procedure  — 

hv  “'j***;  1**^*  involves  a number  of  discrete  phases  or  stages,  linked 

by  decision  n^es  .*  That  is,  each  stage  must  be  completed  before  it  is  decided 
to  undertake  the  tollowiag  one.  Basically,  this  requires;  (1)  separating  develop- 
ment ir^  the  subsequent  production  of  systems;  (2)  first  performing  those  aspects 

« rating  performance,  then  proceeding  to  such  matters 

as  reliability  and  maint aitwbility;  and  (3)  periodic  assessment,  redefinition  and 
readjustTOnt  of  program  cost,  product  performance , schedule  and  the  need  for  further 
technical  advance. 

Such  a procedure  has  long  prevailed  in  consumer  product  industries.  New  drug 
and  chemical  products  cusi^miarlly  require  a "pilot  plant"  stage,  while  many  air- 
craft engines  and  entire  aircraft  (the  Boeing  707.  for  example)  have  benefited 
fr^  a two-stage  process  in  which  the  first  stage  is  a sizable  (non-Incremental) 
military  contract.  Incremental  development  also  has  been  widely  used  by  European 
governoents  to  control  the  development  costs  of  sophisticated  weaponry,  and  a 

* e applied  in  U..S.  defense  procurement  ("fly  before' 

you  buy  } , 


Note: 


* “•  development 

is  also  referred  to  in  Item  4 as  "sequential  lesrning". 
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- > * Stages  in  Increaental  Developaent  - 

Ejihlbit  II-D  describes  the  product  Jevelopnent  process  as  essentially  cois- 
posed  of  six  stages;*  ^ shall  briefly  describe  these  belov.  ^ H 

Stage  1;  " - V > ^ U W-ge'5H|  C 

Ihe  objective  of  basic  reseat^  is  aleply  to  develop  general  kaowlei'ite  vhich 
» ■■7  be  applicable  to  a vide  variety  of  product  types,  perhaps  even  la  a 
very  indirect  maimer;  and  for  a variety  of  industries.  Applied  research, 
'however;  is  directed  toward  developr^t  of  a specific  elaas  or  tjp-*  of 
prodcct,  and  essentially  determines  the  technical  feasibilitv  of  tf-at  pro- 
duct. _ . : ^ 

• • » ' ^ -r  . r*  ^ I ■ * 

.•J 


Stage  2; 


Basic  product  planning  is  perfonied  after  the  results  of  applied  research  in- 
^cate  the  technical  feasibility  of  a potential  product,  and  proceeds  to 
determine  its  general  coiMercial  feasibility*  Should  the  product  * vs  this 
test  it  proceeds  to... 


Stage  3; 


Prototype  development,  which  will  now  determine  the  manufacturing  opera- 
tional feasibility  of  the  product*  Ihiring  this,  many  unanticipated  questions 
. of  design,  construction,  test,  producibility  and  customer  evaluitlon  are 
faced  for  the  first  time.  Again  to  quote  a key  reference*,  a prototype  Is: 

••a  full-sized,  or  nearly  fuU-slzed,  model  that  can  be  tested  in  the 
true  physical  envlroamenc  in  vhich  the  final  product  will  be  used... 

It  will  generally  be  built  with  a minimum  of  capital  equipnent.  *•  (and) 
can  be  either  a subsystem  or  a collection  of  subsystems,  depending 
upon  the  needs  of  the  development  project.  ••  (It)  will  determine  the 
feasibility  of  a design  and  resolve  the  uncertainties  surrounding  its 
performance*  To  a lesser  degree  it  will  tend  to  resolve  the  cost 
uncertainties  surrounding  the  manufacture,  .*•  (which  awaits)  actual 
construction  of  the  capital  equipment  needed..*** 

Stage  4; 

If  the  project  now  passes  .another  comprehensive  program  evaluation,  wc  move  to*  • 
Stare  S: 

Detailed  program  planning.  The  first  step  Is  to  prepare  the  complete  final 
design.  Then,  a detailed  examination  ic  made  of  all  steps  necessary  to  pro- 
duce and  market  the  product.  That  must  be  done  in  considerable  detail,  to 
ensure  that  the  production  and  marketing  organizations  can  handle  the  task 
In  the  time  required.  It  is  a truism  that  any  sound  estimate  of  cost,  and 
therefore  decision  on  price,  must  be  based  on  a valid  master  plan  of  event.  * 
Obviously,  such  operating  plans  must  be  predicated  on  lira  design  and  speci- 
fications* 


Bote:  * Selected  readings.  Item  4,  page  22-23* 
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Staj^e  6; 


Ooce  a detailed  inaster  plan  has  been  prepared  on  the  basis  of  the  final  design » 
a valid  decision  nay  be  ncde  to  proceed  with  the  start-up  of  production  and 
•ctivities*  We  then  nove  into  the  start— up  of  prodnctionp  vendor 
procureseut  and  narketing^  soaetljus  teraed  the  **adninistrativ«  leadtiae**. 

Oae  iaportant  phase  of  this  is  tl^e  drvelopaent  of  *V>rkina  drawings”  necessary 
for  the  production  and  tooling  operations. 

Experience  shows  that  we  auat  be  quite  clear  about  the  distiactioo  between 
^product  developn^nt”  and  the  ”developnent”  of  working  drawings.  Ihe  foner  re- 
^ • final  design  and  specifications  with  proven  capability  to  produce, 
latter  nerely  requires  engineers  fitm  the  production  organixatlon  to  eaploy 
those  plans  and  specifications  during  the  administrative  leadtiat  to  develop  the 
detailed  drawings  and  other  working  materials  required  for  fsll-sca.Te  production. 


* Ihe  Effect  of  Concurrency  - 

In  Exhibit  n-E,  the  upper  section  illustrates  the  essential  results  of  a 
fairly  orderly  incremental  procedure^  ih  which. the  activities  of  a basic 
design*  development  of  firm  plans  axui  apecifications*  and  preparation  for  produc- 
tion are  generally  sequential  with  little  or  no  overlap  between  then.  Fbr"  pro- 
duction activity  there  is  somewhat  more  overlap^  but  this  is  restricted  to  func- 
tions in  which  the  preceding  activities  do  not  affect  the  prodocticii  functions 
concurrently  underway.  In  fact*  some  overlap  between  the  first  three  activities 
could  also  occur*  provided  that  there  be  no  results  in  one  of  them  which  CH>ald 
affect  earlier  results  in  a succeeding  activity. 

i 

* f 

Each  activity  has  some  degree  of  cross-hatched  area  in  It.  IMs  roughly 
*^*P^®**®^*  ^he  fact  that  some  of  the  work  must  be  - discarded  because  it  represents 
the  results  of  false  starts*  correction  of  errors*  etc.  Such  costs  are*  of  course, 

for  product  development  and  introduction.  ^ 

Ihe  lover  chart  indicates  some  effects  of  concurrency*  due  to  more  overlap 
between  the  various  stages  than  in  the  upper  chart.  The  white  area  on  the  second 
roughly  corresponds  to  the  total  area  on  the  first  for  function.  Bov- 
ever*  there  is  now  shown,  by  cross-hatching  in  a different  direction,  certain  added 
costs  for  each  function.  These  new  costs  may  appear  even  in  the  basic  design 
activity  as  a result  of  ha$te  and  shortcuts  taken  to  oieet  the  tight  deadline.  In 
addition*  starting  the  development  of  firm  plans  and  specifications  before  the 
basic  design  is  completed  (with  appropriate  tests)  Increases  the  dount  of  second 
phase  work  obsoleted  by  the  errors  normal  to  basic  design  effort.  The  need  to 
meet  an  urgent  production  deadline  requires  extra  effort  in  the  same  tiaespan  and 
this  condition  cascades  into  subsequent  activities,  increasing  the  amount  of  mis- 
takes and  rework. 

The  final  result  of  the  concurrency  is  that  production  does  not  have  a rell— 
available  In  a timely  manner.  Bence  its  choice  of  metho<ls,  processes, 
vendors  and  tooling  is  unreliable,  as  well  ms  any  plans  for  manpo%ier  and  material 
buildup  and  use.  In  turn,  this  unreliability  infects  the  choice  of  production 

Pl«ot  layout,  production  sequences*  mix  of  worker  skills,  and  balancing 
of  production  lines  and  work-in-process  requirements.  The  degree  of  these  effects 
is  much  increased  by  substantial  interactivity  among  drawings  and  among  production  ^ 

processes*  as  well  as  between  production*  procurement  and  design  activities. 
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Many  typea  of  disruption  cost  aay  be  important  in  a given  circumstance. 
Other  than  labor  hours » some  five  main  effects  of  design  delays  and  changes 
generally  require  close  attention:  increased  labor  rates » greater  overhead, 
extra  material  cost,  financial  coats  and  cost  impact  on  other  projects.  How>- 
ever,  such  a abort  discussion  as  this  cannot  hope  to  cover  them  all  vith  any 
degree  of  thorooghneas.  Ue  shall  therefore  focus  our  attention  on  labor  hours. 


3.  TbttllatililiB  of  PlTBPtioii  to  Labor  Honni 


Concurrency  disrupts  production  labor  through  the  design  delays  and  changes 
which  occur.  Their  presence  indicates  s degree  of  immaturity  in  product  design 
which  leads  to  unpredictable  effects  on  production. 


Efficient  production  demands  specific  disciplines  concerning  schedule, 
methods,  tooling,  manpower  application  and  material  receipts.  Proper  balance  in 
these  matters  creates  momentum  and  rhythm  in  the  production  process.  Their  ab- 
sence creates  frustration  and  a general  malaise  in  the  production  process.  Design 
uncertainty  is  an  efficient  tool  for  destroying  production  discipline.  Many  plana 
cannot  be  completed  and  those  made  cannot  be  implemented,  while  the  work  that  can 
be  started  is  continually  interrupted  and  coopllcsted  by  unpredictable  changes. 


* Design  Delays  - 


.1 


The  delay  of  most  interest  to  us  occurs  near  the  beginning  of  production  or 
after  it  starts;  a delay  well  before  production  is  due  to  start  permits  reschedu- 
ling or  even  cancellation  at  minimum  loss.  Significant  delays  may  suspend  or 
defer  shop  or  vendor  activities,  perhaps  causing  layoffs  with  associated  loss  of 
learning.  They  may  delay  hiring,  creating  imbalance  in  a carefully  planned  build- 
up of  skills  aod  personnel  assignments,  extending  the  period  of  training  for  new 
employees  aod  generally  preventing  the  organization  from  reaching  the  "rhythm** 
necessary  for  efficient  rcrfoimance. 

Since  delay  requires  the  shop  and  vendors  to  ''work  around**  the  problem,  it 
forces  revision  in  production  sequencing,  lhat  creates  interference  vith  other 
work,  greater  worker  movenent,  idle  time  and  perhaps  revisions  to  methods  and 
tooling.  These  consequences  dilute  the  effectiveness  of  shop  control  procedures 
and  complicate  material  planning  and  inventory  records.  Frequent  shop  delays  in- 
crease day-to-day  variations  in  workload,  reducing  a supervisor's  ability  to 
measure  the  effectiveness  of  his  subordinates  and  hence  control  them,  and  causing 
expensive  duplication  of  setups. 

* T 

Compounding  all  this  is  the  shop's  need  to  accelerate  its  work  once  the  de- 
layed design  is  received  using  long  hours,  overmanning  and  inefficient  methods  to 
minimize  the  delay.  The  results  might,  on  occasion,  be  impressive.  But  the  cost 
is  always  excessive  and  the  organization  is  whipsawed  as  it  shifts  back  and  forth 
from  underuse  to  frantic  effort. 


- Design  Instability  - 

Even  changes  in  product  design  that  occur  well  before  production  starts 
can  be  disruptive.  For  example,  a major  growth  in  design  and  introduction  of  a 
radically  new  production  process  may  require  serious  revision  of  the  company's 
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orjductiob  Forther,  any  change  to  one  Major  cooponent  or  subsysten  Might 

effects  on  others,  with  other  consequences  chat  are  hard  to  predict. 
All  this  disrupts  both  the  design  organization  and  the  procurcMenc  process. 


Design  changes  that  occur  after  production  begins  have  Many  additional 

they  often  delay  other  work,  creating  the  delay  effects  des- 
cribed^ove.  Second,  they  cause  direct  loss  of  learning,  since  a task  already 
ht'^***S  replaced  by  a new  one.  Third . the  personnel  affe^ 

4^**^*”  '•««»•  Thl*  puts  then  out  of  phase  with  others  working 

alOTgside  t^.  often  slowing  those  others  as  well;  at  best,  an  inefficient  tn- 
alignaent  of  iMrkload  nay  be  required. 


Fonr^,  design  changes  nay  require  changes  in  production  Methods,  which  in 
torn  require  new  tools,  faculties  or  labor  skUls.  This  nay  also  require  extra 
training,  caraing  further  delays.  Fifth,  a change  Involves  variable  and  non- 
cccurri^  labor  tasks  for  different  units  (since  they  depend  on  the  stage  of 
conpletira  reached)  which  are  ir.herently  aore  expensive  to  perfora.  Further, 
rawrk  or  retrofit  of  units  already  fabricated  or  assenbled  generally  is  done 
^cr  lafavorable  conditions  because  the  affected  unit  has  proceeded  to  a new 
location  not  ao  suited  to  the  work. 


change  costs  are  difficult  to  estlnate  because  the  changes  arc  intri- 
cately involved  with  other  work  and  require  revised  sequencing  of  subtasks  and 
conplex  teardown  and  rework.  Barely  is  there  any  conparable  experience  available 

- toually  the  primary  source  of  estiaating  accuracy. 

U1  this  Makes  budgets  unreliable,  which  weakens  nanagenent  control.  Seventh,  it 
~ ®‘**“  Inposslble  to  segregatu  the  actual  costs  of  making  a change.  Even  if 
tora  can  be  done,  nany  changes  require  a vast  number  of  cost  accounts,  conplicating 
•'•P*^^*ory  task  and  providing  numerous  excuses  to  workmen  for  poor  perfor- 
mance. Bence,  introduction  of  many  changes  weakens  floor  budgetary  control  over 
*****^*fl®B  koaks,  even  If  budgets  can  be  accurately  estlaated. 


El^tb,  a design  change  may  produce  growth  to  the  product  Itself,  increasing 
the  prodwtlon  task.  This  becomes  disruptive  if  it  delays  production,  conflicts 
s^stantially  with  the  company's  planning,  requires  nore  Manpower  or  facilities, 
c^ges  methods  or  affects  supplier  deliveries.  And  ninth,  many  changes  dlrecUy 
affect  the  cost  of  performing  other  work  which  itself,  theoretically,  is  not 
chMged  at  all.  The  change  and  its  rework  requirements  nay  affect  the  sequence  of 
a large  nwber  of  operations,  the  avallabUlty  of  certain  facilities  and 
critical  skills,  supervision  or  storage  space. 


It  is  also  true  that  numerous  "snail"  changes  are  likely  to  be  disruptive. 
For  each  one  still  requires  extensive  paperwork  and  review  by  many  responsible 
people  la  the  design,  production  and  procurement  organizations.  One  small  change 
can,  for  example,  affect  a single  part  which  appears  on  many  drawings,  or  which 
Is  used  is  many  places  on  the  product. 


“ Isplications  and  Ripple  Effects  - 


There  Is  an  isportant  further  consequence.  Taken  togetherg  cost  increases 
due  to  disruption  produce  a sizable  increase  in  the  production  task.  This  is 
true  even  when  we  exclude  those  task  increases  produced  by  design  growth.  It 
Might  be  said  that  such  task  growth  iSg  in  a real  sense,  •’nomal”  when  fixed 
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production  schedules  are  established  on  the  basis  of  an  unst  Me  deaigu.  Once 
the  production  process  is  set  In  notion.  It  is  not  possible  to  delay  and  chanee 
designs  %rithout  expending  additional  labor  and  other  costs. 

ThuSi  the  plant  finds  itself  operating  at  aanpouer  levels  veil  above  those  anti- 
cipated in  designing  the  facility,  planning  the  workflov  or  selecting  equlpaent  and 
tooling,  with  a grwing  inefficiency  which  coapounds  the  cost  of  other  disruptive 
events.  The  various  fonts  of  task  growth  interact  anong  tbenselves,  as  the  confu- 
sion bred  by  delay  and  change  feeds  on  itself.  Orderly  planning  and  control  proce- 
dures take  tine  to  perform  and  quickly  faU  behind  the  pace  set  by  rapid  changes 
to  product  design  or  production  nethods,  or  the  need  to  recover  timm  lost  by  design 
delays.  The  disruption  taxes  supervision  to  the  point  that  major  in  organi- 

zation and  size  become  necessary,  bot  it  always  seems  to  take  a long  timm  to  locate, 
and  train  good  supervisors.  In  the  meantime  the  labor  force  continues  to 
grow,  the  exiting  cadre  is  spread  even  thinner  and  efficiency  deteriorates  further. 

The  degree  of  production  disruption  may  be  somewhat  mitigated  by  relaxation  of 
the  delivery  schedule  to  accommodate  the  delay  or  change.  When  cannot  be  done, 
a "time  coopresslon"  effect  can  multiply  tie  disruption  cost,  as  described  further 

♦ . . . 

‘ - ’-r*  ‘ 

^ Time  Compression  — 

Time  compression  (TC)  occurs  when  an  organization  has  insufficient  time  to 
perform  a prescribed  task  with  the  resources  available.  Given  those  resources,  TC 
is  the  result  of  an  Imbalance  in  the  ratio  of  time  to  task  through  having  either 
.less  time  or  more  task.  TC  can  affect  any  type  of  organization:  design,  produc- 
tion, marketing,  administration  or  a combination  of  these.  The  urgency  of  meeting 
a schedule  dictates  additional  costs  in  order  to  recover  lost  time  or  to  expedite 
matters.  Kore  generally,  TC  creates  cost  by  changing  the  time-cost  trade-off  rela- 
tionship and  by  creating  confusion  and  inefficiency. 

For  example,  TC  may  result  from  the  decision  to  shorten  a product *8  delivery 
schedule  because  a customer  insists  on  tighter  delivery  cycles  than  normal  or  on 
accelerating  an  existing  schedule,  or  because  the  manufacturer  wishes  to  capture 
new  business.  Then  again,  TC  may  result  from  uude res tlma ting  of  the  development  or 
production  task  to  be  performed,  a reduction  in  tooling  or  procurement  time  allow- 
ances or  the  desire  to  Increase  product  quality  or  performance  by  design  Improve- 
ments td.thout  allowing  more  time  for  delivery.  However,  given  standard  products, 
experienced  manufacturing  organizations  can  estimate  their  production  costs  and 
time-cycles  with  considerable  accuracy.  Since  it  is  then  easier  to  resist  compe- 
titive pressures  for  unrealistic  delivery  date,  they  rarely  experience  TC. 

As  «#e  have  seen,  concurrency  of  design  and  production  sharply  Increases  the 
amount  of  labor  expended,  which  constitutes  a task  increase.  Without  schedule 
adjustment,  therefore,  concurrency  creates  time  compression.  This  Intensifies  the 
effect  of  cost  Increases  directly  due  to  design  delays  and  changes.  In  the  develop- 
ment phase,  the  inevitable  design  difficulties  directly  Increase  the  task  to  be 
performed,  while  delaying  its  completion.  A similar  process  affects  production. 

The  design  delays,  nibble  away  at  the  time  available,  while  both  delays  and  changes 
increase  the  labor  task  in  ways  ve  have  described.  Time  compression  occurs  on  both 
sides  of  the  tiae/cost  ratio.  Costs  are  further  Increased  by  efforts  to  recover 
the  lost  time  and  to  perform  the  added  tasks  without  further  delay  In  delivery.  > 
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Technle.lly,  eon*  disruption’ due  to  design  delays  and  changes  would  occur 

* r*r*i'*^  uncertainty,  with  only  the  Increment  caused  by 

!o!*^  **  f*”*'  rurther,  once  an  organization  embarks  on  a malor  prolecC 

' I *****  *’^  •**lny»  in  any  area  must  be  Incorporated  expeditiously  and  the  tlM  ^ 
losse.  recovered,  merely  to  minimize  dUrup^n  of  oSer  ar^.' 

»t  i-a*  IW.’Vi  .s,f*  »x3  V. 

®^*^"^8  th^  WsrnpiloB  PrJce^,  1 ^ 

^;"nit  matabnity  i',. 

...  du"“^v^^^e«^’*o^  nTsS;;:T5:! 

three-dlBenslonal  nodel  also  has  bam  h <*«8ign  events.  An  anlaated 

> ' visoalixlng  the  convolutions  which  Is  of  considerable  help  In 

The  flowchart  la,  however,  a useful^L^f^  *“'*  Pr^uctlon  processes  are  involved 
costw  employing  computer  modeline  technf  ^ detailed  study  of  disruption 

lncorp^te^er“:«  LHsoiSf.  "*  to 
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- The  normal  Cost  Pattern  - ' " 

f L - . f ^ ^ j. 


MEASURDJC  labor  disruption 

1.  B^lc  Measuring  Techniques 

. \*  * »'  ••  -f'.  , r.  I 


f #•• 


technIlogJ^aid‘'tt«i”wi^*«^L,'th2LrconflM*'‘t<**’““-r®  advanced  In 

As  a resSt  they  Involvr^n^cJJu  nr!dLMl  ‘“  “^“f^turlng  processes, 
a cost  Improvement  pabtem.  Exhibit  III  A kjJ!  ***rfc*^f°***’  **  their  manhours  follow 
*stlc  and  logarithmic  sclle  «rfr  k ***  ‘’**^*  P«tem.  both  on  arlth- 

a decreasing  rate.  The  second  Indlcates't^li***  ”*****  reduction  of  unit  hours,  at 
when  plotted  on  logarithmic  scale^  and  InolLe^!^***  *P‘***‘‘*‘'‘^*  ‘*>i*  tost  pattern 

between  each  pair  of  un^  whf « « ^ f * ““sts"t  rate  of  cost  reduction 

Given  a so^.d'^calculaSroJ  '**  *”* 

linear  Isprovement  i^orgreat  d'tect^rJL”"'  "’’!  **^*^*“0*  “f  log- 

amount  of  disruption  costs.  ^ tectlng  the  presence  and  measuring  the 

to  ex!:^n'the®I^jl!ct‘’ff‘Ss^p“ve  eve^i^^sTl'T'^h"*  P^^ioolsrly  well-suited 
development  of  technl,Ls  tHiJculin  the  cosf  of  T’  **“  encouraged 

been  known  to  practltLners  of  the  rr  .«%  ^ ***  *''*"‘*‘  ““T  of  which  have 

described  these  tectoJ;!;^  at  lI^L  “T!*"  f **^'"“00  <2) 

plications.**  * ’ * number  of  refinements  and  new  ap- 


Notes: 
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See  pp.  285-396. 
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When  a significant  change  occurs  for  the  costs  of  a task  which  follows  the 
iaprovenent  curve,  a substantial  •'blip"  wUl  appear  on  the  regular  downtrend  of 
the  cost  line.  TTie  blip  occurs  because  soae  work  has  been  forced  to  return  to 
unit  one,  so  the  improvement  gained  on  the  original  task  has  been  lost.  The 
awnmt  of  penalty  reflects  the  aaount  of  the  old  task  deleted  and  that  of  the  new 
substituted.  Tether  application  of  this  pattern  to  labor  turbulence  is 
described  elsewhere.*  . 


Another  iaportant  disruption  event  is  the  interruption  of  production,  which 
generates  penalties  upon  the  resunption  of  output.  The  traditional  technique  of 
evaluating  this  involves  a '^retrogression'*  approach,  in  which  the  penalty  if  cal^ 
simply  by  assoaing  that  the  entire  task  retrogresses  on  the  inprovenent 
sequence  by  a sufficient  number  of  laiits  to  produce  a cost  increase  of  the 
appropriate  amount.  Subsequent  units  would  then  proceed  from  that  position  in 


the  sequence  in  the  usual  fashion. 


/ . rigid  cost  fomnla.  It  is  more  likely  that  a variety  of  recovery  patterns  occoi, 

some  more  and  others  less  rapid,  as  indicated  by  Esdiibit  III-B,  v.«ich  displays 
a range  of  patterns,  from  a rather  sharp  "scallop"  shape  for  a rapid  recovery, 
to  a very  slow  recovery.  Selection  of  the  initial  peoUty,  the  duration  of 


penalty  and  the  shape  of  the  recovery  pattern  are,  of  course,  matters  for  cart- 
ful evaluation.  Recently,  the  appropriate  calculations  were  reduced  to  a compu- 
ter program  to  facilitate  the  estimation  of  these  iaportant  disruption  effects. 


. f . 


The  Conflict  of  Scheduling  with  Crew  Size  - 
The  substantial  size  and  sophisticated  design  of  many  advanced  technology 


products  also  means  that  considerable  space  is  required  to  build  them.  This  makes 
material  handling  expensive  and  places  a prcmiia  on  carefully  planned  material 
handling  techniques.  It  also  encourages  the  use  of  sizable  crews  of  skilled  people 


who  often  move  unit-to-unit  more  than  the  units  move  crew-to-crew.  Hence,  the  de- 


velopment of  production  methods  and  sequencing  is  an  important  part  of  the  planning 
process,  and  stable  decisions  here  are  essential  to  achieving  reasonable  costs. 
These  procedures  are  strongly  affected  by  substantial  amounts  of  design  delay  and 
change.  In  order  to  measure  the  effects,  we  must  first  understand  the  elements  of 
effective  scheduling  in  the  1C  environment. 


The  isgirovemeot  phenomenon  means  that  holding  crew  sizes  constant  generates 
a continual  rise  in  production  rates.  Thus,  if  the  customer  requires  a constant 
delivery  rate,  shop  scheduling  is  seriously  complicated.  One  solution  - to  make 
periodic  reductions  in  crew  size  - is  damaging  to  production  efficiency.  The 
other  - to  buildup  a large  inventory  of  finished  units  - will  allow  meeting  sche- 
dule during  the  early  period  of  low  output  but  require  extra  time  and  substantial 
investment. 

• 

This  source  of  planning  complexity  and  financial  strain  is  compounded  when 
fabrication  and  assembly  activities,  for  example,  each  operate  on  different  im- 
provement slopes.  Tbe  relative  cost  reduction  from  one  unit  to  the  next  then  is 
different  between  workcenters,  and  they  get  out  of  phase  as  one  accelerates  more 
rapidly  than  the  other.  A similar  effect  occurs  when  a workcenter's  rosltion  in 


1. 


Hotes;  * See  Reference  (2),  pp.  333-366. 
**  Ibid,  pp.  367-396. 
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COST  EFFECT  OF  PRODUCTION  INTERRUPTION 
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the  production  pipeline  places  various  crews  at  different  positions  on  the  unit 
8e«|uence  at  a given  tiae.  It  also  happ*>ns  when  the  output  of  sone -vorkcenters 
is  aeasured  by  a different  unit  sequence  than  that  of  others • due  to  varying 
usage  of  each  workcenter*s  output  on  the  end--unlt* 


v'i  ‘ V 

'*1 


■-•1 


It  is  thus  difficult  bur  essential  to  establish  work  schedules  and  nanpower 
budgets  with  two  characteristics:  (1)  relatively  stable  nanloading;  (2)  main- 
taibing  the  cost  iJBprove»ent  trend  through  acceleration  of  output  rate,  which 
■eans  deliberate  and  consistent  shrinkage  of  flowtines  for  the  various  repetitive 
tasks.  This  requires  careful  activity  analysis  of  the  construction  process  in 
order  to  develop  specific  workflow  directions  for  each  crew.  The  lack  of  e mov- 
ing production  line  makes  i..xh  schedule  disciplines  of  critical  importance. 


■ H 


Activity  analysis  deteraines  the  basic  flow  of  work,  other  than  that  per- 
formed on  a converyorixed  or  other  such  internally  disciplined  system.  It  aims 
at  determining  the  most  efficient  sequence  of  major  production  events  in  view  of 
the  tooling,  facilitic^s  and  space  available.  To  do  this,  it  oaist  assume  well- 
defined  methods,  and  focus  on  broad  man-machine  relationships  and  sequencing  of 
major  production  steps.  ^ . 

Specific  move  dates  for  major  assemblies  and  components  can  only  be  established 
on  the  basis  of  a soundly  conceived  and  stabla  activity  analysis.  These  move 
dates  are  fundamental  tools  of  coordination  and  schedule  Jiscipline  for  numerous 
crews.  By  permitting  orderly  acceleration  of  output  rates,  they  allow  achieve- 
ment of  cost  Improvement  budgets  while  minimizing  changes  to  crew  sizes  and  assign* 
ments.  . • % 


**  The  last  step  in  the  procedure  - the  workflow  chart  - requires  solid  accom- 
plishment of  the  first  two.  It  comprises  the  establishment  of  a carefully  defined 
work  plan  for  each  crew*8  activities  which  itemizes  all  individual  tasks  and  sche- 
dules them  in  an  efficient  daily  sequence.  It  then  assigns  people  to  each  task 
to  ensure  stable  work  assignments  and  conformation  with  established  methods. 


These  three  steps  are  closely  interwoven  and  directly  dependent  on  well-esta- 
blished methods,  facilities  and  plant  layout.  That  requires  plans  and  specifica- 
tions for  the  product  to  be  firmly  set  well  before  production  is  scheduled  to  start. 
Thus,  many  changes  to  design  cause  serious  loss  of  labor  control  which  is  com- 
pounded by  the  penalty  costs  which  accompany  task  changes,  personnel  transfers  and 
production  interruptions.  \ 


2.  The  S-Curve 


We  shall  now  discuss  a method  of  formall/  estimating  disruption  costs,  pro- 
posed in  1959^^)  and  developed  extensively  since  then.  This  method,  known  as  the 
••S-Curve”,  has  been  found  applicable  to  a variety  of  production  situations  which 
involve  long  cycles  for  production  labor,  the  same  conditions  as  found  for  the 
Improvement  or  ^learning”  curve.  An  example  given  by  Exhibit  III-C  will  be  dis- 
cussed later. 


- Disruption  and  the  Improvement  Curve  - 


The  improvement  curve  is  sometimes  used  by  mass  production  or  “short -cycle** 
industries  to  predict  the  costs  of  Introducing  a new  design  into  production.  For 
various  reasons  (shortened  cycle  times,  shallow  slopes,  low  significance,  account- 
Ing  problems),  the  improvement  phenomenon  in  considered  to  disappear  during  later 
stages  of  production. 
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For  loo^cyclo  prodocto,  bovroer^  th^,  oituotloa  i»  sore  coi^lex.  Labor  laam- 
lag  oo  acw  prodocto  coatinuaa  aiaoct  indefinitely  la  a log-linear  pattern,  vblc|i 
reflect'*  a rbytbaic  inproveaent  Inoolriag  six  factora.*  However,  Introdactloa  of  a * 
new  pxoouct  cauaeo  additional  coat  above  tba  aoi'^  iapxoveaent  trend*  Tbe  effect 
oo  direct  labor  boon  la  aaall  for  relatively  stable  deaigna  and  nomal  aetboda  bat 
Bucb  greater  vbea  tba  new  prod^  requires  aijor  aetbods  ebanges  vitb  new  tooling, 
extensive  caployee  trafrOng  and  a protrac«ad  period  of  startup*  Ibis  Ispact  increases 
vbea  tbe  design  Is  unstable  and  has  to  be  debugged  sad  changed  darlng  tbe  prodne* 
tion  process,  and  again  by  design  delays* 


Clearly  tbe  S-Qirve  bulge  is  geourated  by  the  costs  associated  vitb  design 
dftlays  and  dhanges*  T'ae  of  these  costs  can  br  calcslated  by  carve  aetbods 

bat  aaay  involve  interactions  not  so  easily  capto*  4* 


Jr 


■"1 


Tbe  diamptioM  goes  well  beyond  the  first  few  mits  produced*  In  a 
8-Oirve  is  a decay  pattern  for  dismption,  and  li*  e going  on  a diet  to  shed  Cst,  the 
prccess  Just  teltea  tine*  Tbe  way  production  starts  has  nucb  influence  on  a ^ sequent 
units*  Men  work  by  ^xai^le,  and  tbs  proc'?dures,  and  aetbe  Ss  established  darf . g 

tbe  first  year  car*,  .oivard  to  sobsequent  peri^^  and  nuits* 


Design  growth  and  changes  require  revisions  to  production  aetboda  aad  sequencing, 
and  in  faciliticc  asage^  When  such  revisions  are  introduced  after  prodnetloa  pa^’tems 
are  established,  it  t^iee  aoch  tine  sod  several  unita  to  iapleaent  then,  plus  tbe 
learning  losaes  oo  those  units*'  Msoy  design  changes  occur  sft**r  tbe  affected  oonpo* 
nenta  or  operations  are  already  cospleted,  requiring  ripout,  rework  and  reins  tails  tiou* 
Failures  of  no#  coaponents  nl^t  require  renoval  and  rework  of  like  itcas  fron  sub* 
sequent  units,  and  iLXer  re  installation  of  a replacenent*  Intecie  efforts  to  ninl* 
nize  delivery  delays  require  shifting  people  U.  tbe  aost  urgent  tasks,  so  that  pipe* 
lines  are  drained  an^  foHoie^m'vork  neglected,  causing  re-start  costa  for  subsequent 
unita. 


t 


Tba  tine  conpreaalon*  leads  to  naopower  transfers  from  one  task  to  another  and 
back  again,  to  relatively  inefficient  catch-up  procedures,  to  ovex^-nanning,  to 
use  of  inexperienced  people,*  to  videapread  overtine  and  so  on.  . The  for  such 
practices  of  tea*  continues  to  foUov^on  units  in  order  to  offset  the  delay  effects 
of  disruptive  events.  Hbc  i nany  vnlts  are  in  varying  stages  of  ooiipletion,  yll  are 
•ffaoted  by  the  disruptions  operative  at  the  tine.  Uork  in  process  la  held  up,  ami 
tb  inventory  accunulatloo  denonds  frequent  rearranging  to  locate  itens  currently 
required  for  the  shop,  sad  other  drable  handling  that  affects  all  units  in  process. 
The  cloggins  up  of  valnable  staging  areas  and  workspace  Interferes  \rfth  work  on 
follow-on  units. 


Ind  finally,  tbe  cost  increases  generated  by  disruption  severely  increase  the 
denands  oo  nanagenent,  atretehing  its  capacity  and  re*  ling  labor  efficiency.  It 
tine  to  work  out^  this  effect*  Even  when  new  nanagers  are  acquired,  labor 
efficiency  nost  suffer  during  a considerable  training  p*  *oi. 


Rote:  * See  Reference  (2),  page  63. 
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- Size  acul  Duration  of  the  halge  «- 


The  "size**  of  the  S-Curve  bulge  may  be' defined  An  term  of  its  Initial  height 
above  the  underlying  linear  inprovenent  curve/  Its  ••duration"  refers  to  the  nuaber 
of  units  over  which  it  continues:  Bulge  sise  can  differ  due  to  the  intensity  of 
the  problems  caused  by  design  concurrency.  Bulge  duration*  is  linked  to  bulge  size 
to  som  extent,  since  a greater*  size  inherently  takes  longer,  to  work  itself  out. 
Recovery  of  the  original  trend  line  must  proceed  in  a credible  fashion  ^ one  cannot 


expect  ridiculously  sharp  slop^. 


However,  other  factors  are  laportant  in  determining  the  duration.  One  is  the 
need  to  produce  ••enough^^  units.  For  example,  the  adequacy  of  tooling  and  metliods, 
and  the  need  for  their  correction,  arc  determined  partly  by  the  evaluation  of  mis- 
takes made  or  problems  discovered  during  productico.  Hence,  a sufficient  nuid>er  of 
units  must  be  produced  before  all  start-up  problems  can  be  detected  and  so  resolved. 

A similar  consideration  applies  to  the  disruption  caused  by  design  delays  changes. 
The  discovery  of  some  design  problems  occurs  only  as  units  have  a change  to  progress 
through  all  aspects  of  test  and  even  into  field  operations.  • 

. .Sti  H 

However,  adapting  to  circunstances  of  new  product  introduction  is  also  a mat-  V 
ter  of  tine.  For  example,  many  new  typ^  of  tooling.  Jigs,  fixtures,  work  sequen- 


cing,  control  procedures  and  personnel  skills  must  be  absorbed  by  and  into  the 


existing  organization.  Related  training  and  other  activities  take  time  to  organize 
and  conduct,  as  do  the  incorpozmtloa  of  correezions  in  tooling  and  facilities. 
Similarly,  the  discovery  and  correction  of  design  defects  requires  careful  study 
by  engineers,  the  progress  of  which  relates  more  to  the  capacity  of  the  desioa  or- 
ganization than  tc  the  cumulative  output  achieved.  The  production  organ! zat loo 
also  takes  time  to  replace  inefficient  procedures  initiated  to  work  around  a delay 
or  to  absorb  the  inpact  of  an  urgent  change. 

^ .. ; - i / - i i 

The  close  relationship  of  bulge  size  to  the  degree  of  concurrency  described 
briefly  by  Reference  (2)  as  “time  coapressioo“  - led  to  a description  of  the  bulge 
**  * cost".  Given  a "standard**  shape  for  an  S-Curve  of  a given  slope 
it  is  then  useful  to  describe  the  bulge  size  in  terms  of  a "TC-ratlo"  with  a value 
of  unity  for  the  standard  curve.  While  the  degree  of  TC  Itself  cannot  be  meamed, 
Its  impact  on  a production  organization  c^n  be  described  and  its  final  cost  effect  •*" 
measured  In  terms  of  the  amount  by  which  actual  hours  exceeded  the  base  should-cost. 
Therefore,  a manufacturing  facility  that  experiences  TC  repeatedly  can  develop  em- 
pirical methods  of  predicting  its  cost  effects.*  In  doing  so,  the  degree  of  TC  for 
a new  product  can  be  viewed  in  terms  of  the  adequacy  of  the  "leadtlae**  available  to 
develop  and  Introduce  that  product  into  production. 


pie  question  bolls  down  to  determining  the  degree  to  which  several  main  oper- 
ating functions  can  be  done  efficiently  In  the  tine  available.  Knowing  the  factorr 
which  control  the  performance  of  each  function,  we  can  study  perfomance  on  past 
products  and  relate  that  experience  to  the  degree  of  disruption  cost  which  occurred, 
expressed  as  a TC-ratio.  Then,  for  a new  program  having  specified  Icadtime  charac- 
teristics, we  can  predict  its  TC-ratio,  from  which  the  bulge  cost  itself  may  be 


- Calculating  S-Curve  Patterns  - 

In  recent  years,  computer  techniques  have  been  developed  to  permit  easy  ad- 


Jusoient  of  the  S-Curve  bulge  for  three  conditions:  (1)  the  slope  of  the  underlrlng 
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linear  Inproveioent  trend;  (2)  the  degree  of  tlae  coopresslon  effect  or  bulge  sizc» 
as  measured  by  the  TC-ratlo,  and  (3)  the  number  of  units  produced  before  the  beige 
phases  into  the' linear  cost  (bulge  duration).* 


An  example  of  S-Qirve  patterns  for  a large  product  is  charted  by  Exhibit  m-D. 
It  is  based  on:  (1)  an  83  percent  linear  cost  trend,  (2)  an  estimate  that  a 25  per- 
cent TC-ratio  is  required  to  allow  for  the  labor  cost  effect  of  major  new  tooling 
and  facilities  on  the  start-up  of  a new  product,  and  (3)  a projection  that  the 
atart-up  effect  will  last  for  six  units,  in  view  of  the  time  required  to  work  out 
the  resulting  problems.  The  first  curve  above  the  linear  trend  (marked  by  TC  of 
• 15)  shows  the  distribution  of  mit  costs  for  a more  modest  (and  more  usual)  start- 
up expense.  Other  curves  show  the  effect  of  Mgjher  TC-ratios  on  Initial  cost  »d 
on  bulge  duration. 


3.  ruTther  Confirmation  of  DisrmgrtM  lOBicaai 


J 
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As  stated  earlier,  it  is  rarely  practical  to  identify  r measure  individual 
costa  of  labor  disruption.  Thus,  in  the  past,  the  main  means  of  measurement  has 
involved  broad  comparison  with  "as-built  should-cost"  calculatioos,  using  S-€Uzve 
techniques  to  forecast  the  broad  inpact  of  concurrency  on  labor  hours.  However, 
with  deeper  understanding  of  the  disruption  phenomenon,  we  can  now  use  three  amaly- 
Clcal  te^inlques  to  demonstrate  the  consequences  of  concurrency  in  some  detail* 
These  are:  variability  analysis,  network  analysis  and  computer  simulation. 


The  greater  detail  involved  in  such  analyses  provides  more  precise  testing  of 
our  understanding  of  the  phenomenon,  while  allcvlng  us  to  prepare  more  accurate  es- 
timates of  concurrency  costs.  They  also  have  the  interesting  - and  perhaps  signi- 
ficant - qiwlity  that  the  further  you  go  with  any  of  these  analyses,  the  more  ser- 
ious you  find  the  consequences  of  concurrency  to  be. 


- Variability  Analysis  - 

Vell-knowh’ terhnlques  of  measuring  the*  effects  of  uncertainty  have  been  used 
for* some  years  tinder  the  heading  of  queuing  theory.-.  These  clearly  show  the  effect 
of  random  variation  In,  for  example,  the  time  a specific  operation  can  start,  or 
the  time  required  for  its  performance.  Such  variability  demonstrably  Increases 
costs  for  sequential  operations  unless  suitable  inventory  or  float  is  provlded- 
lk>wever,  this  Involves  extra  Investment  and  increases  the  need  for  rework  as  a re- 
sult of  a design  change.  These  effects  are  much  increased  as  more  operations  are 
involved,  or  for  large  complex  products  which  have  limited  room  for  float. 

Such  conclusions  clearly  apply  to  a situation  affected  by  the  unpredictable 
design  delays  and  changes  .^hlch  result  from  concurrency.  For  they  cause  Irregular 
delay  of  start  dates,  unexpected  changes  In  task  sizes  and  injection  of  rework  and 
retrofit  at  unplanned  locations. 

— Network  Analysis  - 

This  simply  establishes  three  basic  facts  or  estimates  for  each  major  task  in 
a project.  In  approximate  order  of  importance,  these  are:  precedence  relationships, 
task  duration  and  resources  required. 


Note:  * The  original  data  for  TC  *■  1 and  a duration  of  70  units  appeared  in 

Reference  (1)  and  Reference  (2),  pages  160,  170  and  18A.  Subsequent 
computer  refinements  have  altered  the  unit  of  measure  in  which  they 
are  expressed,  but  not  the  shapes. 
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As  a 8l&(»Ie  example,  a model  was  devlbcd  to  describe  a major  steel  fabrication 
activity  “ t jrJcer  hull  conatructlon.  It  clearly  demonstrates  the  severe  Impact  of 
variability  from  schedule  in  the  start  dates  of  hull  modules,  of  interruptions  to 
fbe  regular  flow  of  work,  and  of  a modest  increase  in  the  cost  of  one  major  cob(k>~ 
rent  due  t«i  design  changes*  « 

• w . . . • V . 4.'  r . 

The  model  was  built  on  fourteen  characteristics  of  hull  construction*  The 
first,  eleven  of  these  were  used  to  run  a benchmark  simulation  involving  no  variabil- 
ity of  module  start  dates  from  their  normal  flo%rtlme  requirement,  or  of  labor  per* 
fonaance*  Subsequently,  the  last  three  conditions  were  added  to  reflect  the  impact 
of  design  delays  and  changes*  As  a result, . there  was  an  expansiom  of  41  percent  in 
the  completion  or  "launch"  date,  and  a cost  increase  of  22  percent*  Since  the'  de-  . 
sign  cost  increase  was  eight  percent,  the  net  disruption  cost  was  fourteen  percent* 
No  provision  was  made^to  minimize  schedule  slippage,  which  would  introduce  further 
cost  penalties. 

- • - ..  »rl  >^v  . 
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— Practical  Labor*  Estimating  - - 


. (. ; 


*-  t ^ • .-lii 

. ^ 4 x'*' 

Construction  of  a complete  simulation  model  for  major  new  program  is  a time  ■ ^ 

consuming  task  and  requires  much  detailed  data,  even  when  restricted  to  labor  hours* 
So,  while  such  models  should  be  constructed  where  cost  and  time  permit,  it  is  de- 
sirable to  have  a more  economical  ^proach  to  the  prediction  of  disruption  effects* 

For  example,  a simpler  model  can  incorporate  more  traditional  estimating  procedures, 
applied  with  specific  consideration  of  the  effects  of  design  delays  and  changes, 
while  allowing  rapid  adaptation  to  a variety  of  conditions  for  use  in  'Sihat-if" 
exercises. 

An  important  objective  is  to  determine  the  degree  to  which  potential  design 
delay,  design  changes  and  time  compression  force  the  production  operation  to  devi- 
ate from  its  original  plan  (presumably  that  plan  has  been  matched  to  the  plant's  * 
capacity  and  operating  procedures)*  Hence,  the  first  step  in  the  simulation  is  for 
the  user  to  provide  such  data  as  the  schedule  of  production  deliveries,  basic  flowr- 
times,  the  initial  cost  level  for  each  major*  departxEtent  or  task,  and  their  labor 
input  patterns,  major  precedence  relationships  between  tasks  (a  siaplifled  network),  - 
and  the  Improvement  slope  to  be  followed. 

The  computer  uses  this  data  to  develop  and  display  a basic  manhour  plan  for 
the  project,  which  shows  the  progress  and  labor  hours  for  the  project  as  it  would 
be  without  design  uncertainty.  After  making  necessary  adjustiaents,  the  user  pro- 
ceeds to  enter  his  forecast  of  design  delays  and  changes,  and  their  disruption 
effects.  An  interactive  computer  procedure  provides  appropriate  guidance  in  en- 
tering a number  of  important  Judgments  regarding  the  effects  of  design  delays  and 
changes  on  labor  hours.  This  allows  him  to  assess  the  reasonability  of  his  entries 
in  relation  to  each  other  and  the  end  result,  and  to  adjust  them  as  necessary* 

The  model  already  offers  promise  as  a practical  tool  for  testing  hypotheses 
as  to  the  causes  of  known  amounts  of  disruption  cost,  and  for  estimating  potential 
disruption  effects.  As  greater  experience  is  gained,  it  will  be  expanded  to  pro- 
vide for  tlaie-phased  labor  rates  and  costs  of  overhead  and  material. 

IV.  EVALUATING  THE  QUALITY  OF  MANAGEMENT 

' ' • T 

Up  to  now,  we  have  mainly  addressed  the  disruption  produced  by  concurrency 
of  design  and  production  to  meet  a tight  delivery  date.  TV^o  other  sources  of 
major  disruption  - force  majeure  and  inadequate  planning  and  management  - have  not 
been  covered  in  any  detail. 
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Xhe  oveniheljBiiig  nature  of  force  ujeure  extensive  discussion  here  In- 

appropriate.  The  term  itself  denotes  "an  unexpected  and  disruptive  event  which 
may  operate  to  excuse  a party  from*  a contract"*.  Literal  translatism  yields  "su- 
perior force"^  which  suggests  the  need  to  bow  to  factors  or  events  of  substantial 
power  not  under  one's  control.  For  exasplsp  natural  disasters^  war^  strikes^  etc. 
are  essentially  beyond  rational  prediction.  Such  events  are  subject  only  to 
United  protection^  as  by  insurancep  or  to  evaluation  in  terns  of  long  range  trends. 
It  would  notp  for'exaaple^  be  prudent  to  expand  one's  investment  in  a country  known 
to  be  headed  for  revolution^  except  under  circunstances  offering  unusual  protec* 
tion  and/or  rewards. 


The  responsibilities  of  nanagenent  in  a disruption  process^  however^  requires 
our  close  attention.  The  quality  of  nanagemoit  can  affect  the  outoome  of  a devel* 
opMot  project  in  many  ways.  Equally  iaportant,  the  uncertainty  of  product  devel* 
opnent  can  severely  Unit  the  power  of  the  nanagerlal  function^  so  that  major  con- 
currency of  development  and  production  virtually  pre-eapts  good  planning  and  adnin- 
istration. 


1.  Key  Managerial  Ptoctions 


Kanagers  inevitably  become  steqied  in  the*  skills^  the  knowledgie  and  the  lore 
of  the  industry  in  which  the  enterpries  foactions.  Hsny  good  managers  are  also 
experts  on  their  industry^  or  in  eons  sperfal  aspect  of  it.  Hevertheless^  that  is 
not  their  basic  function  as  sudi.  It  is  indeed  the'manager's  responsibility  to 
ensure  that  he  has  the  facts  needed,  and  he  must  be  able  to  decide  whether  he  is 
actually  getting  them.  This  requires  some  degree  of  knowledge,  else  be  would  be 
taiable  to  coaHuiicate  his  questions  or  absorbs  what  his  advisors  are  saying.  But 
once  an  "appropriate"  level  of  technical  background  is  obtained,  h-ig  function  is  to 
cvploit  it  by  actions  designed  to  advance  the  relevant  interests  of  the  organization. 


Ikider  one  classic  definition,  the  nanagenent  process  may  be  characterized  by 
five  functions:  planning  (obviously  toward  an  objective),  organizing,  staffing, 
directing  and  controlling**.  This  description  does  not  identify  two  characteristics 
crucial  to  this  discussion.  First,  is  the  constraintB  Inposed  by  time  and  physical 
resources  (human,  physical  and  financial).  The  manager  must  rhink  la  terms  of 
developing  a plan  that  is  possible  of  attainment,  which  entails  evaluating  the  re- 
aourcea  and  time  available  to  be  used.  Organizing  miat  similarly  be  concerned  with 
the  resources  available  in  the  form  of  staffing,  and  even  with  the  capability  avail- 
able for  directing  and  controlling.  In  abort,  resource  and  time  limitations  envelop 
the  entire  management  process  with  a web  of  interacting  relationships,  which  must 
be  considered  meticulously.  * 

Farther,  In  performing  his  tasks  the  manager  must  determine  whether  a situa- 
tion contains  hidden  problems,  and  what . their . real  souixes  may  be,  using  the  limit- 
ed infoxMtion  which  may  be  gained  or  analyzed  in  the'  time  available  to  him.  The 
critical  importance  of' the  maiuger'a  problem' identification  and  analysis  activity 
has  been* acutely  described  as  follows*^. 


’ Hote; 


ass 


Random  House  Dictionary,  unabridged,  1973.. 

See,  for  exaaple,  "Principles' of  Management"  by  Koontz  and  O'Donnell, 
McCraw-HULl.  . Also  a modified  version  in  "Management"  by  P.  F. 
Drucker,  Harper  A Row  1974,  pages  393  and  400-401. 

"The  Kational  Manager",  Kepner  A Tregoe  1965,  2nd  Edition  1976,  p.  39. 
Privately  printed. 
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"rte  rw  aaterlAl  of  aanagewnt  !•  InfonaCloa... about  the  uorld  about 
hm,  about  hla  orgaolzatioo  and  its  plana,  about  the  perfornance  of 

<he  organlxat Ion  where  be*  la  responsible  for  carrrlns 
out  those  plans,*  about  people  iu»d  things  and  conditions.  • .Be  aust  be 
In  recognizing;  before  the  fact,  what  inforaatioe  will 
^ rel^t  and  Inportant  and  what  will  not  as  he  is  in  recognizing. 

_ i^th  the  benefit  of  hindsight,  the  obvious  relevance  and  isBortanca 
of  certain  infoxxutionl** 


and  again*: 


“ --  <>  I ' - 

* ‘ . * ■ * .*  - * y.*' r i ^ -*s 

» -1 

r'.  mmimgfmKat.  gitwlxic  pcogreaaluely  won  eao,lex.  and'e^rleac.  «orc 

. ' obaolet.  More  rapidly,  th.  aaugpr  mat  rely  sore  and  aoce  on  ridllfol. 

^nestionlng,  and  less  and  less  on  experience.** 

i ^ ^ ..  - • 

• It  is  that  nanageaent  decision-making  requires  a considerable  degree 

of  abstraction  (as  opposed  to  withdrawal)  from  dair-to-day  events.  Only  in  this 
way  can  ^ nanager  nove  into  the  resin  of  ideas  in  which  logical  analysis  can  lead 
^ effective  dedsion,  without  which  he  cannot  resist  cnotiooal  insolvenent  and 
t^  bias  of  his  experience  and  associates.  This  is  not  to  any  that  things  necesa-  • 
mmt  be  analysed  natheaaticaUy  or  on  a conputer.  The  process  is  nore  like 
what  happens  when  a scientist  is  groping  for  a "fruitful-  hypothesis. 

Drucker  ^tacks  the  point  from  a sonewhat  different  direction  - that  of 
a decision  • but  the  essetkce  is  sinilarl: 

"A  decision  is  a judgnent.  It  is  a choice  between  alternatives.  It  is 
• choi^  between  right  and  wrong.  It  is  at  best  a choice  between 
alnost  right  and  *probably  wrong*  - but  nuch  sore  often  a choice  be- 
tween two  courses  of  action  neither  of  which  is  provably  nore  nearly 
right  than  the  other. 

* s 

-...nanagers  who  make  effective  decisions  know  thst  one  does  not  stsrt 
with  facts.  One  starts  with  opinions.  These  ere,  of  course,  nothing 
but  untested  hypotheses  and,  as  auch,  worthleaa  unlesa  tea  ted  against 
reality.  Th  determine  what  is  s fact  requires  first  a decision  on  the 
criteria  of  relevance,  especially  on  the  appropriate  measurement.  This 
hinge  of  the  effective  decision,  and  usually  its  smst  controver- 
sial aspect. - 

The  natural  tendency  for  mmagers  to  become  set  in  their  ways  of  decision  making 
U especially  dangerous  in  the  presence  of  rapid  change.  At  that  time,  old  base- 
lines are  shifting  and  the  accustomed  categories  of  thought  are  becoming  irrelevant, 
/jnd  new  technical  training  and  disciplines  may  be  essential  to  function  efficiently 
in  the  new  environment.  The  need  for  abstract  thinking  is  radically  intensified, 
suggests  that  the  manager  be  trained  in  a variety  of  basic  dicipllnea,  to 

bia  adaption  to  constantly  changing  technology.  • Paradoxically,  he  must  be 
s in  basic  information  and  knowledge ' in  order  to  handle  the  extreme 

specialism  of  today's  world. 


Rotes:  * Ibid,  page  23. 
l # Op.  cit.,  pp.  470-471 
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2.  The  Inpact  of  Dncertalnty 


T. 


It  !•  connionly  presumed  that  any  major  disaster  (except  force  majenrc)  signl- 
flea  poor  management*  But  that  fails  to  recognize  that  some  degree  of  imcertaincy 
Is  inherent  In  events*  It  also  ignores  the  important  degree  to  which  managers 
operate  tinder  severe  time  and  resource  constraints,  and  must  be  selective  regarding 
the  information  they  seek  and  the  degree  to  which  they  evaluate  It  prior  to. taking 
action.  Thus  even  .first-class “managa^t  .cannot.be  infallible,  so  that  disastrous 
events  may  happen  under  its  administration* 

Tb  use  a sports  analogy,  when  two  football  teams  of  top  ranking  meet  in  a bowl 
game,  the  losing  coach  is  rarely  accused  of  poor  management.  This  is  true  even 
though  mistakes  were  made  on  the  field  and  the  winner  may  have  exploited  the  loser's 
weaknesses.  It  is  recognized  that  the  loser  had  to  be  superior  to  get  into  the  gane 
in  the  first  place,  and  that  it  was  almost  inevitable  that  someone  would  lose.  Again 
in  baseball  a batting  success  ratio  of  only  30  percent  is  considered  hi^y  competent 
Drucker  applies  the  idea  as  follows*:  ’ . ... 

^ Nt*.  r " ^ ^ ^ 

^ne  focus  of  the  organization  must  be  on  performance  •••  But  performance 
does  not. mean’^success  every  time.'*  Performance  is  rather  a 'batting  . 
average*...  It  must  have  room  for  mistakes  and  even  for  failures*  Vhat 
performance  has  no  room  for  is  complacency  and  low  standards.” 

Some  degree  of  uncertainty  must  always  be  faced.  In  planning  future  activities  the 
manager  obviously  must  minimize  it;  this  is  one  reason  for  his  chronic  conservatisa. 


~ Balancing  Risk  and  Reward  - 


w ^ 

Now  suppose  the  degree  of  uncertainty  can  be  quantified,  either  by  using  past 


experience  or  by  applying  analytical  techniques  oa  the  basis  of  a supportable  theory. 
We  must  face  the  question  of  how  to  judge  a manager's  decision  to  proceed  when  he 
knows  that  there  is  significant  probability  of  failure.  To  do  so  requires  some 
sense  of  how  such  a risk  is  evaluated. 

In  accepting  risk,  the  terms  of  trade  must  he  clearly  assessed,  and  these  will 
differ  from  case.  For  example,  economic  growth  is  worth 'less  to  seme  managers  (or 
to  their  capital  sources)  than  to  others.  Thus,  arriving  at  a rational  balance  of 
risk  against  reward  requires  an  explicit  idea  of  the  relative  value  of  specific 
losses  and  benefits.  This  makes  the  "utility  function"  a key  item  of  the  manager's 
tool  kit  — the  idea  that  the  financial  consequences  of  a decision  are  not  Just 
proportionate  to  the  dollars  involved.  For  example,  one  Investor  nay  accept  a 
twenty  percent  chance  for  a loss  of  a mUlion  dollars  if  there  is  a ten  percent 
chance  for  a profit  of  five  million  dollars,  while  another  will  not  dare.  The 
terms  of  trade  vary  between  people,  and  even  for  the  same  person  it  will  vary 
over  time. 


Even  within  this  set  of  value  systems  the  risk  of  total  failure  must  be  mini- 
mized by  suitable  plans.  An  important  means  of  limiting  the  chance  of  major  damage 
caused  by  failures  of  individual  tasks  in  a program  is  to  establish  practical  alter- 
"backup  activities.  The  benefits  of  such  redundancy  may  be  expressed 
by  a simple  example.  Suppose  that  three  activities  - A,  B and  C - must  be  coc- 

successfully,  and  that  they  have  individual  success  probabilities  of  P , P 
^C.  probability  of  success  for  the  total  project  is  P - ^ 

For  a value  of  .80  for  each  activity,  this  yields  an  overall  success  probabllxty  of 
P - .83  - .512. 


Note:  * Op.  cit.  pp.  A56-457.  . 567 
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T »!!!*  activity  has  associated  with  It  an  alternative  activity. 

In  other  words,  for  activity  A to  fall,  both  A and  Its  alternative  A’auat  faU. 
Further  assunlng  that  alternatives  M,  V and  C'also  have  probabilities  of  .80.  we 
■ay  co^te  tte  revised  probability  of  success  as  I' - .885.  The  l^troveMnt 
^^Iflcance  when  we  consider  Its  converse,  the  probability  of  project  failure. 

2!  ^ 7 - -^SSto  1 - .885  - .115,  a cut  of  over  three-quarters 

project  were  to  be  doubled  by  the  redundancy,  the  1 
probable  cost  of  failure  would  be  less  than  half.  i-.-.  . . 


SuHlng  In  view  of  the  inherent  uncertainty  he  aust  deal  with,  realistic 
Judgmnt  of  the  aanager  s coiapetence  nust  face  five  main  qnestlons:  (a)  whether 
the  gaiw  waa  worth  the  candle  - was  the  project's  objective  big  enough  to  warrant  . 
the  resources  and  risks  Involved;  (b)  whether  the  alternatives  to  «-M«  action  were 
rationally  defined  and  considered  by  all  concerned;  (c)  whether  the  probable  results  f- 
of  those  ^tematlves  were  as  undesirable  as  failure  of  the  project  under  consider-  , 
atlon;  (d)  whether  suitable  steps  by  which  to  prevent  failure  or  to  bold  its  cost 
to  a alnlma  were  available  and  (e)  whether  the  aanager  provided  adequate  aonltor-  ' 
Ing  of  events  to  detect  dererloration  of  the  sltMtlon  at  mt  early  stage,  and  to  t 
limit  the  losses  (this  also  xeqslrta  planalmg  chose  limiting  steps  In  advance). 


# • J 

^ cannot  fault  m otanager  aerely  because  he  proceeds  In  the  face  of 
possslble  failure;  his  decision  can  be  entirely  ▼alld  whether  or  not  the  consequences 
turn  out  to  be  favorable.  Going  into  a dangerous  situation  with  open  eyes  is  quite 
different  froa  going  in  aerely  knowing  that  it*  s dangerous.  Were  all  the  above 
questions  properly  addressed  before  proceeding*  the  losses  doe  to  an  unfavorable 
turn  of  events  would  usually  be  liaited  sufficiently  to  avoid  disaster.  Equally 
iaportant*  if  that  disaster  ultiaately  did  occur*  it  should  be  relatively  •’accept- 
able to  the  value  structure  of  the  oianager  and  his  sources  of  capital. 


/' 
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- -Facing  the  Unaeasurable 


Unfortunately*  the  degree  of  uncertainty  and  the  potential  effects  of  unfavor- 
able events  are  often  frustratingly  difficult  to  determine.  Many  situations  in- 
volve potential  events  whose  probability  is  not  measurable.  All  too  often  the 
term  calculated  risk”  refers  to  a decision  reached  in  recognition  of  factors  known 
to  be  unfavorable  but  susceptible  to  calculation.  The  future  may  even  involve 
events  of  a totally  unexpected  or  unknown  type  - soaetiaes  termed  "unkncTwn-unknowns” 
or  •’unk-unks”  for  short.  Thus*  in  dealing  with  complex  problems  under  liaited  time 
and  other  resources*  substantive  errors  in  Judgment  can  occur  in  unpredictable  ways. 


For  example*  there  could  be  a shift  in  market  structure  subsequent  to  making 
a major  coaaitaent  for  facilities  or  advertising.  Straws  in  the  wind  may  even 
have  been  there*  but  they  were  simply  not  read  because  knowledgable  people  thought 
the  time  was  past  for  further  study  and  the  hidden  weakness  compounds  to  become  a 
debacle.  It  seems  intemperate  to  describe  such  an  error  as  poor  management  — there 
is  Just  some  irreducible  degree  to  which  things  like  that  will  happen. 


Here*  the  prudent  manager  can  only  defend  himself  by  establishing  stringent 
monitoring  of  all  areas  in  which  a sudden  shift  in  events  might  have  such  a sub- 
stantial impact  (avoid  surprises)*  by  establishing  practical  back-up  alternatives 
in  each  ar^a  (systems  redundancy)*  and  by  being  alert  to  the  need  to  limit  damages 
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mnd  even  abort  programs  as  appropriate  (catastrophe  planning)  • The  I'^licy  of 
"^ainimaa**  (minialxing  the  maxlanai  daMge)  ia  not  a had  vay  of  describing  this. 

* Ibe  Riaka  of  Technological  Change  - 

As  used  here,  the  ten  '^technological  change"  refers  to  two  conditions*  One 
Involves  the  developnent  of  new  engineering  knowledge;  the  second  refers  to  the 
process  of  closing  s major  gap  between  an  orggnixat ion's  ares  of  expertise  on  the 
one  hand,  and  what  an  undertaking  requires  it  to  perfom  on  the  other*. 

Technological  change  involves  substantial  uncertainty  for  the  organlratloos 
affected  and  Its  links  with  nanagenent  process  &re  complex*  There  are  many  practi- 
cal ccmsequences  for  the  declalon-naker.  For  Instance,  a progran  Invol'^ing  coor:- 
cnrrency  of  design  and  production  is  riskier  when  a snail  quantity  of  production 
units  is  involved.  A larger  quantity  provides  the  opp  rtunity  for  a greater  sales 
inoone,  through  which  disruption  costs  night  be  recovered* 

Bapid  technological  change  causes  nanagerial  uncertainty  sinply  because  pre- 
diction has  so  little  to  go  on.  Valid  Judgnents  are  based  on  a blend  of  fact, 
experience  and  cemfimed  theories  which  do  not  apply  fully  to  a substantially 
unknown  technical  situation.  The  effects  are  pervasive,  ranging  fron  the  sudden 
ineffectiveness  of  familiar  technical  skills,  experience  and  operational  proce- 
dures to  the  inpact  of  recondite  scientific  laws*  Further,  a real  technical  ad- 
vance requires  much  knowledge  and  Judgment  from  technicians,  who  are  not  always 
easy  for  the  nanager  to  understand  or  interpret*  The  manager  operates  with  the 
uneasy  feeling  that  there  may  be  buried  somewhere  an  assumption  or  prediction 
which,  under  more  rigid  scrutiny,  nay  p^ve  to  be  both  critical  and  fallacious. 

Under  technological  uncertainty  nany  key  Judgments  cannot  be  made  in  advance, 
and  those  made  may  prove  wrong.  The  entire  program  cannot  be  fully  planned  , but 
must  evolve.  It  is,  then,  not  surprising  that  concurrency  is  costly  for  the  produc- 
tion process.  Great  effort  is  required  to  establish  production  momentum,  and  it 
is  costly  to  shut  it  off  or  slow  it  down  and  then  re-accelerate.  The  production 
organization  is  poor  at  responding  to  the  inherently  sporadic  character  of  a 
development  program,  and  its  floundering  - like  a fish  out  of  water  - is  dreadfully 
expensive. 

It  might  seem  obvious  that  the  solution  is  to  avoid  concurrency,  or  at  least 
to  limit  it  to  areas  in  which  redundant  or  alternative  courses  of  action  offer  op- 
tions by  which  to  avoid  major  disruption.  But  the  matter  is  not  so  simple. 

For  one  thing,  competent  managers  may  be  unable  to  differentiate  between  one 

case  which  contains  the  seeds  of  disaster  and  others  which  don't*  This  often  happens 

in  times  of  rapid  technological  change  when  the  manager's  relatively  limited  techni- 
cal knowledge  requires  that  he  depend  even  more  than  usual  on  his  advisors*  Second, 
the  costs  generated  by  concurrency  are  not  concretely  measurable.  Thus,  they  seem 
less  real  during  the  project  planning  phase,  which  intensifies  the  difficulty  of 
evaluating  the  risk. 

Third,  a period  of  generally  rapid  technological  change  is  usually  one  of 
intense  competition,  since  a major  advance  by  one  enterprise  can  deprive  others 
of  substantial  business.  As  a result,  there  is  great  pressure  to  minimize  the 
time  devoted  to  development  of  new  concepts  and  hardware,  so  that  significant 
development  work  overlaps  with  production.  Fourth,  an  organization  may  be  "forced" 
into  concurrency  to  avoid  an  even  %#orse  situation.  For  example,  older  products 
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May  be  obsolete  and  dying  on  the  vine,  aaking  new  development  of  prime  inportaa^ 
to  maintaining  minimal  return  to  investors,  or  to  avoid  dismantling  a specialized 
production  and  marketing  organization  which  required  much  tiM  and  cost  to  develop. 

Whether  to  permit  concurrency  is  a crucial  managerial  judgement.  If  a proper 
decision  is  to  be  made,  the  issue  must  be  defined  clearly  in  terms  of  the  degree  of 
risk  being  taken  and  the  potential  benefits  from  taking  it.  This  is  mot  easy  to 
do.  Going  to  a more  dynamic  pace  of  product  development  will  open  up  new  maikets 
for  rapid  ezpansioo  at  attractive  prices,  generate  important  cost  reductions  doe 
to  new  processes  or  designs  and  provide  opportunities  to  develop  talented  techni* 
cal  and  marketing  organizations. 

Bowever,  the  risks  are  substsntial  and  hard  to  assess:  serious  technical 
failures;  heavy  costs  of  design,  production  startup,  field  engineering  and  customer 
service;  major  capital  investments  in  production  and  marketing  facilities  - often 
greater  than  predicted;  possible  large  overruns  of  production  cost  targets  due  to 
d^:#ig*i  delays  and  changes;  and  new  marketing  methods  and  personnel  nay  require 
more  time  to  become  effective  than  anticipated,  while  there  may  be  downright  errors 
in  measuring  the  effective  market.  The  manager  must  recognize  that  these  risks 
exist,  define  them  in  detail,  and  measure  their  impact  as  best  he  can  and  provide 
for  them  in  his  comprehensive  planning  and  contingency  analysis. 

3.  Evaluating  Management 

- A Definition  of  Good  Management  - . r « . 

In  defining  good  management  it  is  not  sufficient  to  prescribe  the  five  basic 
functions.  There  must  also  be  suitable  regard  for  resource  constraints  and  uncer- 
tainties plus  adequate  technical  skills  and  knowledge  and  competent  judgment  in 
relying  on  the  information  and  in  using  the  resources  available.  Good  management 
should,  for  example,  minimize  the  chances  of  a major  debacle  in  any  area,  and  its 
effect  if  it  happens.  It  is,  of  course,  best  identified  by  a persistent  record  of 
success,  but  it  will  not  always  be  successful,  reflecting  the  mcer taint ies  inher- 
ent in  its  environment.  Then  too,  in  evaluating  a specific  manager  we  must  allow 
for  individual  differences  in  capability.  Some  men  can  make  things  happen  that 
others  cannot,  while  two  managers  each  may  have  superior  capabilities  in  areas  that 
the  other  does  not. 

The  ’'standard'*  for  performance  is  then  a broad  baseline  defined  by  the  general 
practice  and  level  of  competence  in  a given  industry,  plus  reascmable  anticipation 
of  the  effects  of  outside  events.  Hence  good  management  is  exhibited  by  any  mana- 
ger of  basic  competence  who  attempts  to  recognize  and  work  in  a professional  manner 
within  the  constraints  of  times,  resources  and  personal  capacities  to  meet  his  as- 
signed objectives. 

We  may  further  specify  this  definition  to  say  that  adequate  project  management 
exists  if  it  meets  five  conditions.  First,  the  organization  clearly  has  coiapetence 
for  the  type  of  work  undertaken  (as-contracted).  Second,  it  makes  reasonable  pro- 
vision for  differences  between  prior  work  and  this  project.  Third,  it  has  enough 
capacity  to  handle  all  its  commitments  (space,  equipment,  vendors,  personnel  and 
finances).  Fourth,  it  plans  realistically  the  detailed  steps  required  to  perform 
the  task,  identifies  areas  of  risk,  and  provides  appropriate  back-up  and  bail-out 
options.  Fifth,  it  assigns  suitable  capacity  and  key  personnel  to  the  task,  and 
directs  and  controls  their  efforts  in  a reasonable  way. 
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"Poor**  management , then,  results  from  the  clearcut  absence  of  one  or  more 
of  the  Itcy  characteristics  of  good  managcoent.  A marketing  disaster,  for  example, 
may  reflect  poor  work  in  assessing  consumer  demand,  reliance  on  Inadequate  sales- 
men, superficial  planning  or  lack  of  follow-up.  Some  cost  overruns  occur  through 
inadequate  cost  estimates,  which  themselves  reflect  poor  planning. 


w 


• Performing  the  Evaluation  - 


The  difficult  task  of  evaluating  management  quality  becomes  core  tractable 
If  we  divide  management  activity  into  three  main  categories.  * . - 

^ ^ \ i‘  * x ‘ .Ji  i* 

First,  is  the  vast  number  of  ^^administrative"  actions  and  decisions;  they 
•aay  require  skill  and  experience  but  they  can  be  performed  fairly  routinely  by  pro- 
isrly  trained  people.  This  includes  handling  the  details  of  production  operations 
or  a marketing  program,  internal  coordination  of  engineering,  marketing  and  manu- 
facturing, the  establishment  and  operation  of  reasonable  policies  and  procedures 
for  production  scheduling  and  accounting,  etc.  Such  natters  as  organization  struc- 
ture, plant  location,  etc.  are  more  difficult,  but  in  general  it  is  possible  to 
arrive  at  rather  clearcut  Judgments  of  managerial  adequacy  for  administrative  acti- 
vities* ' ^ • 

However,  it  much  harder  to  evaluate  decisions  on  major  product  selection, 
long-range  strategies  of  various  types,  or  the  power  of  leadership  to  inspire  a 
major  effort  toward  challenging  new  goals.  Such  major  declsicns  involve  close 
Judgments  on  which  reasonable  men  may  differ,  largely  due  to  the  uncertainty  in- 
herent in  the  projections  on  which  the  decision  is  necessarily  based.  A third 
area  involves  major  product  development,  which  involves  substantial  uncertainty 
in  the  effectiveness  of  the  efforts  undertaken  to  achieve  the  objective  or  in  the 
time  required  to  do  so.  * 

^ It  is  often  difficult  to  reach  a clearcut  Judgment  that  poor  management  was 
responsible  for  a particular  disaster.  Inherent  uncertainty  could  well  account 
for  some  situations,  while  technological  concurrency  will  account  for  many  others. 
Thus,  it  may  be  easier  to  arrive  at  a Judgment  of  good  management  than  it  Is  one 
of  poor  management.  . . .j,  , ' * 
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V.  AVOIDING  DISASTER 

It  seems  inevitable  that  organizations  will  undertake  projects  involving  con- 
current product  development  and  production.  When  this  occurs.  It  is  essential  to 
evaluate  the  risks  and  costs  involved  before:  commitments  are  made.  It  is  equally 
important  to  manage  it  with  reasonable  assurance  of  meeting  the  original  require- 
ments for  delivery,  performance  and  cost.  To  do  either  requires  an  understanding 
of  the  concurrency  phenomenon  in  general  and  how  to  manage  it  in  a given  case. 
Having  covered  the  foimer  topic  in  some  detail  above  we  shall  now  briefly  address 
the  latter,  with  special  attention  to  minimizing  the  chances  for  a real  disaster. 

What  might  be  termed  "preventive  medicine"  involves  four  basic  elements:  pre- 
program analysis,  periodic  performance  reviews,  an  effective  early  warning  system 
and  firm  corrective  action  to  limit  the  impact  of  unanticipated  problems. 

Pre-prograa  analysis  focuses  on  identifying  the  proposed  program's  degree 
of  developoient  concurrency  and  on  forecasting  its  potential  effects  on  cost  and 
delivery.  Such  analysis  is  not  easy  to  develop,  but  it  may  well  lead  to  a sharp 
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expansion  In  the  estioated  cost  and  delivery  time  for  a program  and  second  thoughts 
on  its  relative  desirability.  The  process  requires,  first,  a careful  assessment^ 
f the  ^certainty  involved  in  the  development  work  contemplated,  with  due  regard 

A “**  “ compounded  by  interrelatedness  in 

f production  phases.  This  is  not  an  easy  task.  Extensive  technical 
couMel  Is  useful,  but  the  conclusions  of  technicians  Bn.st  be  supplemented  by  the 
healthy  scepticism  of  administrators  who  have  seen  carefully  planned  technical 
developments  undergo  spectacular  failures. 


ram. 


■ ■■  M 


A second  major  aspect  of  this  review  is  to  determine  the  degree  of  overlap 
or  concurrency  between  the  production  and  development  phases.  Again  this  may 
nc«  be  easy  to  do,  since  this  Judgment  Is  also  predicated  on  the  degree  of  SOAA  ‘ 
required  md  an  estimate  of  the  time  required  to  accomplish  it.  However,  by  using 
conservative  experience,  allowing  for  contingencies  and  providing  redundancy  or 
^kup  in  the  most  crucial  activities,  a reasonable  assessment  is  usually  possible. 
When  this  cannot  be  done  the  program  requires  severe  reconsideration.  . ‘ 

A third  step  is  to  measure  the  potential  disruption  of  cost  and  delivery  whl<* 
unfavorable  events.  This  requires  careful  determination 
of  sho^d-cost  as-built,  which  involves  an  assessment  of  the  degree  to  which  the 
initial  ^figuration  will  undergo  design  changes,  which  naturally  occur  in  a 
disorderly  manner.  The  cost  of  disruption  Itself  may  then  be  addressed  by  develop- 
of  model  which  explicitly  considers  the  impact  of  design  delays  and  , 
erratic  desi^  changes  on  the  total  production  system:  engineering  release, 

J^dors,  tooling,  facilities,  production  labor,  associated  price  levels  and  over- 
h<^  costs  and  the  effects  on  capital  investment.  This  analysis  must  distinguish 

the  fwdMental  cost  of  an  undlscupted  product  from  the  effect  of  design  delays 
And  chCQges  • j 

Completion  of  this  estimate  nay  well  produce  a marked  revision  in  appraisal 
o the  project,  which  leads  to  rejection  and  makes  further  analysis  unnecessary. 

. **  must  then  evaluate  several  managerial  aspects.  Is  the  planning  in 

sufficlMt  detail  to  expose  all  crucial  decisions  and  to  define  the  critical  areas 

conslstenc  with  organisation  capacities 

ss  they  exist  or  can  they  be  reasonably  supplemented  at  appropriate  future  times 
making  explicit  allowance  for  the  demands  of  unpredictable  variation  from  the 
e^cted  course  of  events?  Have  the  characteristics  of  necessary  controls  been 
clearly  defined  and  is  the  organization  capable  of  implementing  them  - not  In  terms 

of  paperwork  procedures,  but  as  concrete  measures  of  organization,  observation, 
reporting  and  decision?  . « -tiou, 

^ce  the  program  has  begun,  periodic  review  of  performance  will  proceed  in 
a m^er  to  the  initial  analysis,  with  the  obvious  simplification  permit- 

ted by  completion  of  the  original  evaluation.  When  no  such  evaluation  has  been 
accomplished,  a similar  comprehensive  checUist  must  be  employed  it  the  perfor- 
unce  review.  In  any  event,  the  latter  will  concern  itself  urgeucly  %rtth  identi- 
^Ing  areas  in  which  the  development  process  Is  generating  potentially  serious 
disruption  not  explicitly  anticipated  earlier.  Given  such  events,  t'nc'next  pri- 
orlty  will  be  to  evaluate  their  impact  and  how  they  must  affect  the  ultimate 
value  of  the  prograa. 

Once  committed,  it  is  difficult  for  an  organization  to  be  objective  about 
such  matters,  and  one  function  of  the  operating  audit  is  to  force  cold-blooded 
evaluation  of  the  consequences  which  =say  evoUTng.  There  is  rarely  a clear- 
cut  M^r  to  such  situations;  no  program  can  be  expected  to  be  without  problems 
and  faint-hearted  men  rarely  accomplish  large. things. 


^ Ifi 


Further,  iaportant  outside  circumstances  may  differ  from  when  the  program  was 
initiated,  so  as  to  change  the  relative  impact  of  the  new  problems.  There  are' 
few  definite  rules  for  such  an  evaluation. 

• I 

The  development  of  an  early  warning  system  for  a program  involving  substan- 
tial uncertainty  requires  establishing  suitable  measures  of  performance  discre- 
pancies to  mam  of  an  emerging  program.  This  relies  heavily  on  Judgments  of  where 
the  uncertainty  is  most  likely  to  develop.  The  enormous  implications  of  such  pro- 
blems make  it  essential  that  these  warnings  be  promptly  transmitted  to  the  highest 
levels  and  that  they  be  carefully  appraised  rather  than  treated  as  merely  another  • 
•tatistical  report.  ContinucKl  use  of  the  disruption  model  previously  developed 
also  may  help  in  detecting  significant  difficulties.  • 

l-*»tly  is  the  matter  of  corrective  action  once  a significant  problem  has 
developed.  Should  it  fall  within  the  range  of  the  original  estimates,  the  appro- 
priate response  should  have  already  been  determined  - at  the  least,  the  risk  %rlll 
been  considered  and  the  cost  found  acceptable.  Perhaps,  however,  external 
c^cinwtances  have  changed  sufficiently  as  to  require  further  review  of  the  situ- 
ation. Should  this  be,  or  should  the  problem  appear  to  exceed  the  original  range 
of  expectations,  urgent  consideration  by  top  management  is  called  for.  The  neces- 
sary actiom  may  be  draconian.  Including  substaatial  infusion  of  new  technical  or 
asnagerlal  resources,  termination  of  the  project,  a merger  or  other  sharing  of  risk, 
or  even  bankruptcy  of  the  company.  The  i]q>ortaDt  point  is  that,  consideration  of  ^ 

such  drastic  steps  must  come  much  more  qnickly  than  for  leas  risky  endeavors.  : j 

>_  i : t.*  ..  ‘ 
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**®  "eeting  by  pressing  last  Blnute  business  and  his  paper  was 
wilvered  by  Hr,  George  U.  Sutherland,  Msistant  Director  Systems  Acquisition  flanaoement. 
Director  of  defense  Research  and  Engineering.  . . » ^ 

'•  • . r- . , — 1-;  - j'-n  ir,n 

The  Banagement  of  ^or  system  acquisitions  In  the  Department  of  Defense  has  been  a 
principal  topic  for  public  discussion  sines  the  early  sixties.  I would  characterize  the  ^ 
general  md  of  the  discussions  to  be  an  expression  of  concern— an  expression  of  dissatls- 
faction  in  the  results  achieved  in  the  conduct  of  Individual  programs.  Ue  could  list  a 
n^r  of  programs  that  have  been  scrutinized  thoroughly  In  full  public  view  and  the  measure 
of  management  achievement  rated  marginal  to  wsatls factory.  ^ y 


environment  tterr  Is  a natural  urge  to  counter  much  of  the  crtticism  by 
desertbing  the  passive  task  of  lanagement  together  with  the  complexities  of  rtsk  and 
un^rtalnty;  conclu^ng  with  the  Judgement  that  the  results  have  been  acceptable  and  those 

qualified  to  be  critics.  Certainly  there  are  many  who 
jshare  this  view,  and  defend  their  conclusions  objectively  and  sincerely.  There  Is  no 

Hi*  managing  major  system  programs  Is  complex,  uncertain  and  tough— those 

ki4  unreasonable  In  their  own  defense.  Equally  reasonable,  however, 

4^!^?  1 management  process  when  measured  by  the  ultimate  results  coopared 

to  initial  estimates  and  goals. 


The  management  process  Involves  an  Interaction  between  people  who  are  generally 
directed  toward  common  goals  and  objectives.  While  the  results  achieved  are  visible  to 
many,  the  Internal  management  process  ard  mechanisms  are  visible  to  few— excluding  most 
If  not  all  those  on  the  outside  looking  on  and  many  of  those  within  the  operating  group 
who  are  directly  involved  and  contributing  to  the  results.  The  output  characteristics 
of  the  manageinent  process  are  not  physical  and  lack  the  material  shape  of  the  normal 
product.  I believe  we  fall  to  give  proper  substance  to  management  as  a discipline  or 
function  In  part  due  to  this  la^  of  material  quality. 


There  Is  a popular  presumption  concerning  management  that  places  the  focus  on  output- 
on  the  results  and  as  they  say  In  the  business  world,  the  bottom  line.  Whatever  the  result, 
management  Is  accepted  to  be  the  cause.  In  the  view  of  those  who  look  to  the  Department 
of  Defense  for  results,  this  presumption  Is  reasonable;  however,  the  matter  of  Inproving 
mults  through  Improvement  of  the  management  process  demands,  more  than  a focus  on  results. 
In  correcting  the  problem,  we  must  deal  with  the  entire  span  of  the  management  process 
starting  with  Input  and  progressing  through  to  output.  The  attention  directed  to  the  formal 
process  of  Interactions  and  decisionmaking  must  be  thorough  from  top  down.  We  mjst  learn 
to  deal  with  management  as  a physical  task  and  understand  the  product  In  mateHal  ten®. 

We  cannot  Ignore  the  Individual  Interactions  and  pass  them  off  as  part  of  an  Informal 
^c^s  of  the  organization.  We  must  recognize  that  fixing  management  problem  1$  a 
^yslcal  process  as  much  as  bending  metal— we  are  shaping  people  things,  we  are  establishing 
wgible  control  techniques  and  procedures  to  cause  the  evolution  of  a product  down  the 
line  from  input  through  output. 
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^ ^ manag^nt  of  aajor  sjrstn  acquisitions  In  the  000  Is  underoolnq 

'*‘2^*  ]J***J"  laqwrtant  as  wll  as  coi«>lex  changes  that  nst  have  support  ai?  * 
^rs^dlng  throughout  a broad  segnent  of  Govenvent.  Industiy  and  aca<Mc  InstltuMons 
These  chugn  are  not  the  result  of  an  Isolated  000  undertaking.  The  base  for  these 

J*  ue  are  no  longer  addressing  an  area  of  nanagewnt  activity  that 

*'*k*^***  being  part  of  the  Defense  donaln.  Todqy  we  share  the  stage  with  ainv 

*1"?  **  "0-  • broader  ter^r  Sd5»nL  « uT^ 

if  ^ r®!*  policy  and  In  the  for*  of  standards  to  aeasure  satisfactory  results 

tehwlorl^  Ilii  « ih*  i°S  15  ***  viewpoint  and  par^aP  * 

te^ylor  as  welt  as  the  lack  of  an  adequate  base  on  which  to  assess  the  quality  of  perfbrw- 

****"  progress  In  wovlng  baiards  an  l^troved  najor  systea  acqulstlon 
pro^s.  This  ^ross  hu  not  cow  through  the  efforts  of  000  acting  In  IsolUlon,  I 
to  see  the  value  of  the  broad-based  effort  to  nake  awanlngful 
chan^  In  the  Mageaent  of  aajor  sysbens  throughout  Govemamt  aid  Industry  The  ourmse 
*’?*’?'*  !?  address  this  process  of  change  In  the  000  In  the  proper  Mrspectl^ 

*****  change  Is  not  based  on  the  Jud^Knt  of  a single 
col  lectlv^f  feet  of  a^  Independent  viewpoints.  It  Is  alsoclear  that  * 

rgriEgrj.iiti.’.gr”  “• 

^ a5***?^  should  not  be  taken  lightly.  Kor  should  we  assuaa  that  chanoe  fa  newacAriitf 

uution  ud  logic  as  the  action  progresses.  I would  be  quick  to  add— we  are  not  deallnn 
In  tte  shor^tera;  the  task  to  be  acco^Ilshed  deaands  knowledge.  pati^c2  <teterafnatl2n 
and  The  tendency  to  status  quo  is  Instinctive  to  aost  people  at  all ’levels  In  ai 

a^Mterlal  proAict,  Ulthln  Defense  we  are  diaeglng  basic  structure  when  we  taaper  with 
the  Rianageaent  process  and  should  not  expect  Instant  agreeaent. 

*!f**  <‘<s«9roeiient  or  resistance  to  change  Is  bad.  Quite  the 
S rwistance  to  change  Is  healthy  since  we  are  noving  from  a known  set 

of  conditions  to  the  unknown.  There  Is  security  In  dealing  with  a known  orocess  evnn 
^wlctin«*^»I*r**^*******  ^rtfalls.  Those  idio  strive  for  change  with  the  vision  and 
^alr^in  ^*“*ble  oust  accept  the  fact  that  the  road 

acceptaS!”  ^ ^ ^ ‘ ^ ^ Prosented  with  the  logic  required  for 

*>***■  «c‘lu<s1tion  Is  a nanagewent  technique  that  Is  product  focused.  We 
♦kT*®***™  * program  aroimd.a  major  product  to  be  managed  by  one  manager  and  Include  within 

"^tbe  elements  essential  to  the  employment  of  the  prod*t 

* system.  The  system  management  concept  Is  not  a new  technloue— the  orocess 

Industrv'af  *!l*  ®®^*"**  Department  and  in’segments’^of 

4^  as  the  OHiplexIty  and  cost  of  Mapons  has  Increased.  The  proorai  nanaoawnt 

have  been  ^veloped  over  the  years  through  application  to  aerospace  sea  and 

the^lxtl^aDiJ5«“?^®M  ^ «rolution  of  the  system  acquisition  process/the  period  Jf 
ue  sIxtiM  ^pears  in  history  as  a major  milestone  in  shaping  the  direction  we  are  now 

wmt"oub1^  T“  ****  **”  ***®  '“"a9e«ient  of  major  system  acquisitions 

m!!.  ****  **•  increased  awareness  of  the  problems  encountered  In  the  manaoe- 

!*•?  Congress  became  Increasingly  Involved  in  the  technical,  cost 
l^te^t^d  programs.  The  media  discovered  new  fields  of  pSwic 
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^K  <ncreased  concern  in  the  ■nagenent  of  major  programs  produced  extensive  public 

management  Issues  accurately.  Perceptions^f 
problem  ww  not  always  founded  on  knowledge  of  the  inner-workings  of  the  mMiagemt 
P™ces^  There  was • howver,  agreement  that  the  probles  as  manifested  in  the  results 
achieved  in  various  programs  required  strong,  positive  action  to  iaprove  the  mwageKnt 

Pl^OCftSS* 

From  the  vanUge  ^int  of  hindsight,  we  might  agree,  with  some  debate,  that  the  single 
■osf  important  ac^on  in  establishing  a foundation  for  i^iroving  the  major  system 

legislation  to  create  the  Comaission  on 
eowr^t  P«>a»e«ment.  I telieve  this  action  token  by  the  Congress  and  the  work  of  the 
to^ssion  to  to  a bwchmar*  of  substance  and  we  must  not  allow^  values  of  these  e^ts 
to  to  overlooked  or  to  recede  into  dusty  shelves  as  so  many  have  in  the  past. 

b»  ifc.*"eiih!Jru*'roL'^*'V^®4*  i'ishly  Wropriate  observation  made  in  testimony  given 

Oiairman  of  the  Presidenfs  Blue  Ribbon  Defense  Panel  before 
« Coatee.  Deceaher  1971.  Mr.  Fitzhu^i  noted  that  when  studies 

are  coapleted  and  gwrittee  members  depart,  those  who  remain  behind  to  assess  and 

•*«"  stepper*  on  1"  the  findings -there 
•"  «<iwcate  to  remain  behind  to  give 

sup^rt  to  the  findings  with  understanding  and  conviction  idien  the  meahers  ooto>1ete  their 

I believe^.  Fitzhugh's  coiments  are  va1id~in  the  process  of  change,  there  aist 

®"  fiPveiTOmnt  Procurement  is  meaningful  and  has 
major  ralw  in  guiding  the  direction  of  change.  The  value  of  the  Report  is  not  in  the 

J®  *”  ®f  «*P'<sse<*  viewpoints  and  reconaendations . but  • 
«J*f^i**  ****  Co«lssion  completed  a stu(iy  that  is  current  and  relevant  and  quail* 

TkTo^L!!!®  ■»*“■»>«»»«"*< « and  best  focus  on  the  subject  of  major  system  acquisition. 

**?'*.!*  " effective  foundation  for  change  and  we  should  co^nue  to 

to  drtlL'SJ  toSSiS  iSue1r“  notwithstanding  the  views  we 

M Cimlv  M09.  'Major  System  Requisitions.*  issued  5 Rpril  1976.  effectively 
towslates  the  findings  of  the  Conaission.  dealing  with  major  systems,  into  Federal 

diracM  Ir«  to  A-109  as  a major  statement  of  policy  and  has 

dmeted  ito  attention  to  implementation.  In  the  ioplementotion  of  the  A-199  policy. 

TOre  mist  be  a clear  understanding  of  issues;  an  appreciation  for  the  comlexity  of  the 
and  recoyiition  that  there  are  no  sinplistic  solutions.  The  actions  directed  to 
oTthiTwillT^'^suaMT***  ^ conducted  with  knowledge  and  persistance;  anything  short 

S^equmt  to  the  Report  of  the  Covission  on  Government  Procurement  there  have  been 
sewral  related  ODD  studies  directed  to  specific  areas  dealing  with  the  management  of 

'*“^'^el_^1oi$ition  Review  Comnittee  (RMARC).  1973-197^  the  Navy 
»d  tonne  Acquisition  Review  Committee  (NMARC),  1975;  various  Air  Force  studies'  and  the 
A^utoltion  Advisory  Gn»up(AAG)  in  1975.  Indiiiduali;  and  collwMvelT  SIT C^"sio^^ 
R^rt  and  the  internal  000  studies  provide  a valuable  body  of  information  directed  to 
tounSion”*"^  system  management.  The  actions  being  taken  by  Defense  rest  on  this 

AcMon  by  000  to  proceed  with  the  i^>lenentat1on  of  changes  to  the  major  system 
acquisition  ^cess  was  Initiated  formally  in  early  1976.  Di^ion  was  given  to  the 

Engineering  to  Implenient  selected  reconmendatlons  submitted 
Group.  While  the  Initial  action  was  taken  In  response  to  the 
MG  report  Is  inportant  to  understand  that  the  overall  planning  for  change  and  the  actions 
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Issued  in  Aprtl  1976.  While  the  AAG  Report  was ’dlrecterto'a's^^lfic  and  ii.iteo 
segment  of  the  process,  the  recomnendations  support  liportant  areas  of  policy  In  A>109. 

Chmges  planned  and  underway  In  000  Involve  specific  areas  of  the  major  system 
*^5  ^■<’®rtant  to  recognize  we  are  dealing  with  change  to  a well 

established,  existing  process  that  does  not  bend  or  adjust  with  ease.  The  task  Is  not  to 
revolutionize  the  process;  the  task  Is  to  make  specific  adjustments  and  brfno  the 

Into  alignment  with  the  new  policy.  The  areas  of  change  are  well  defined 
and  the  mrk  of  restnicturInQ  the  process  undenm^. 


' The  principal  areas  of  change 
process  InvDlve  the  following: 


1. 


^ 2. 


3: 


In  terms  of  the  linpact  of  the  new  policies  on  the  DOO 

• * ■*  I “** 

Creating  the  role  of  the  Defense  Acquisition  Executive  to  provide  for  the  ■ 
Integration  and  unification  of  actions  In  the  management  of  major  system 
acquisition.  . 

' • ■ V.  ■ 'i/  rvr 

Restructuring  the  front-end  of  the  major  system  acquisition  process  to  establish  ■“ 
a new  Secretary  of  Defense  decision  point  based  on  mission  need.-  : v 

^centralization  of  selected  program  review  actions  to  the  Service  SecreUrIcs  i 
la  support  of  the  Secretary  of  Defense  declsloamaklng  process.  -.it-- 


4.  E^bllshlng  a management  methodology  for  the  development  of  a planned  acquisition 
Strategy  tailored  to  Individual  programs. 

In  discussing  these  key  Issues,  it  is  not  the  purpose  here  to  Justify  the  change  but  rather 
to  present  the  changes  as  Inplemented  In  the  Department  of  defense  and  the  goals  and 
objectives  of  the  new  policy.  ^ 

Defense  Acquisition  Executive  > ^ :V.V-T  '.V 


The  basic  responsibility  of  the  acquisition  executive  as  set  forth  In  /V-109  policy 
is  to  Integrate  and  unify  the  system  acquisition  process  and  to  morltor  policy  impleiwita- 
tlon.  Certainly,  this  function  Is  a major  role  of  the  acquisition  executive  Is  expanded 
to  a larger  role  than  envisioned  In  A-109;  the  Defense  Acquisition  Executive,  In  addition 
to  the  responsibilities  Included  In  A-10>,  Is  the  principal  advisor  and  staff  assistant 
to  ^ Secretary  of  Defense  In  matters  pertaining  to  system  acquisitions.  The  Importance 
Of  ^Is^ltlon  Is  seen  In  the  supporting  functions  performed  by  the  Acquisition  Executive 
in  ^ decislorjTOklng  process— he  provides  a central  role  In  the  Office  of  the  Secretary 
of  Defense  (OSD)  to  focus  the  several  system  program  elements  In  the  context  of  a total 
program  and  attempts  to  bring  the  best  balance  In  developing  recommendations  for  the 
Secrotai^  of  Defense.  As  Chairman  of  the  Defense  System  Acquisition  Review  Council  (OSARC), 
he  has  formal  access  to  the  corporate  thinking  of  the  Secretarial  Staff,  and  Is  able  to 
pursue  separate  functional  viewpoints  to  the  depth  necessary  In  establishing  the  best  total 
pro^«  action.  In  this  manner,  he  performs  a role  similar  to  that  of  the  Deputy  Secretary 
of  Defense  In  the  previous  organization  arrangement. 


cross-function  perspective  Inherent  to  the  role  of  the  Defense  Acquisition 
Executive  has  been  a missing  role  In  OSD  below  the  level  of  the  Deputy  Secretary.  The  OSD 
staff  organization,  t^lcal  of  many  large  corporations,  provides  the  essential  fiaictlonal 
expertese  in  the  vertical  staff  elements  but  lacks  the  strong,  central  mechanism  to  present 
a ^nagenent  viewpoint  that  cuts  horizontally  across  the  staff  in  presenting  the  best 
solution.  Typical  of  staff  actions,  cross-staff  positions  develop  throu^  the  process  of 
coordination  and  Invariably,  through  conpromlse  of  strong  staff  positions  to  achieve  a 
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coimon  ground  for  agreement.  In  the  face  of  tm  opposing  positions » the  oofnproolse  Is 
acceptable  but  not  necessarily  the  best  total  program  action. 

The  Defense  Acquisition  Executive  should  provide  the  ability  to  Improve  the  decision- 
making  process  In  support  of  major  system  acquisitions  by  forcing  functional  staffs  to 
assess  their  separate  positions  In  the  context  of  the  total  program.  Trade-offs  within 
and  between  elements  should  develop  as  more  balanced  program  actions  rather  than  one-sided 
functional  actions.  Good  system  program  decisions  require  thoughtful  compromise  between 
staff  elements  In  the  real  sense  of  the  corporate  interest  to  achieve  the  best  program 
results.  . ,L  ‘ I . 

: 'X  jir  * ‘ 0. 

The  single  most  Important  change  to  the  DOD  major  system  acquisition  process  to  result 
from  the  A-109  policy  Inv-olves  the  restructuring  of  the  front-end  of  the  process— U«ose* 
activities  that  occur  prior  to  the  full-scale  engineering  development  decision,  starting 
with  the  request  to  Initiate  a new  major  system  acquisition  program  and  progressing 
through  to  the  Identification  of  a system  proposed  for  full-scale  development.  In  this 
phase  of  the  system  acquisition  cycle,  occur  the  most  fundamental  decisions  and  actions 
In  shaping  the  program  to  acquire  a solution  to  meet  a Defense  mission  need.  It  Is  In 
this  phase  idiere  we  have  the  greatest  opportunity  for  error  In  creating  the  basic  condi- 
tions for  a successful  or  unsuccessful  program  as  ultimately  judged  by  the  results 
achieved. 


Restructuring  of  Front-End 
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Some  will  argue  this  view  to  be  an  overstatement  of  the  Impact  of  early  program 
actions  on  the  final  results.  I do  not  believe  so.  There  Is  much  evidence  to  show  that 
the  Impact  of  the  early  program  actions  Is  vital  in  determining  the  quality  of  the  solution 
selected  to  meet  the  mission  need;  the  technical,  financial,  contractual,  logistical  and 
production  characteristics  of  the  program;  and  the  degree  of  risk  and  uncertainty  intro- 
duced in  the  decisionmaking. 

In  implementing  the  new  ^ront-end  policy,  the  Department  of  Defense  has  established 
a new  decision  point  identified  as  Milestone  0,  the  Secretary  of  Defense  decision  to 
initiate  a major  system  acquisition  program.  This  first  action  by  the  Secretary  to  consid- 
er the  procurement  of  a new  capability  Is  directed  primarily  to  determining  the  essential 
nature  of  the  stated  mission  need  and  the  reconciliation  of  the  need  to  existing  Defense 
capabilities,  priorities  and  resources. 

The  Initiatives  to  be  taken  leading  to  the  Milestone  0 decision  rest  w1t|i  the  Heads 
of  the  Components  and  In  the  case  of  the  three  Military  Departments,  w^th  the  Service 
Secretaries.  The  DOD  policy  requires  that  the  Services  establish  procedures  for  the 
continuing  analysis  of  Defense  idsslon  areas  to  Identify  mission  needs  under  certain 
conditions,  however,  the  Secretary  of  Defense  will  also  request  the  Service  to  take  an 
Initiative  to  pursue  a solution  to  a mission  need.  Needs  will  be  described  In  relation  to 
segments  of  a mission  area  and  Identified  as  mission  elements  corresponding  to  the  perceived 
need  for  a new  capability.  A nission  element  Is  not  a preconceived  segrrient  but  exists 
only  as  the  definition  of  an  Identified  need.  The  scope  of  the  element  Is  determined  by 
the  Service  to  be  however  broad  or  narrow  as  the  Service  may  define  the  mission  job  to  be 
done.  The  principal  Issue  Is  that  the  mission  element  must  describe  the  job  to  done  in 
terms  of  the  eneny  situation  and  not  In  terms  of  a system  to  accomplish  the  job.  The 
'iscusslon  of  a mission  need  separate  and  ap^rt  from  the  system  to  perform  the  task  is 
new  to  the  DOD  and  Is  not  an  Inherent  ability  of  the  existing  functional  activities  In 
the  Services  or  OSD.  There  is  considerable  learning  Involved  in  the  full  Implementation 
of  this  concept  and  we  would  not  limit  our  view  of  the  change  In  terms  of  existing 
organizational  functions  since  It  may  be  necessary  to  Introduce  new  functional  groups  and 
techniques. 
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_ tin®  thst  ft  visslon  n6€d  Is  Idftiitlflftd  ftnd  ft  dstftnnlnfttlofi  kv 

tervice  ^ftt  ft  nm  cdpablllty  Is  required  to  meet  the  need,  the  Service  will  prepftre  a 
^ SUtenent  (HENS);  a key  docuoent  Introduced  ty  000  to  oooMiute 
^ **!*  ^*”*^*'>  Defense  and  requesting  approval  of  the  need  and  the 

s<"V>y  •««»  without  delay.  The  HENS  serves  tte  sane 
0 declsioiwiklng  process  as  the  Decision  Coordinating  Paper  (OCP) 
a^I^4M  ®**"‘»*  milestones  1,  II  and  III.  Defense  Systs  ' " 

Acquisition  Review  Council  action  will  not  be  involved  at  Nllestone  0. 

the  pertaining  to  the  nisslon  eIcKnt  need. 

c.p*m^,’’psss:';2;„':.srs  wS'.nf!  x^si;  L"pSl?s 

Jftte  HLJ  KT^  lISjiS?  * “•  “™"*  " “•  >»'« 

InfoiTMtlon  essential  to  the  Initial 
decision  with  the  principal  focus  on  wisslon  need,  to  Include: 

- the  ene^y  situation  presented  in  finite  terns, 

- ^ tine  ^Hod  of  the  need  during  idilch  the  capability  will  be  anloyed. 

^ ^ attacked  and  associated  tactics  einloyed  by  the  enew  fortxs 
■ <«««^f1cat1on  of  eneevl^tS.  iiHaS^S  * 

^Ung  ^ hostile  environnent  In  which  the  systen  will  operate  to  accoapllsh 
wift  mtssiOfi  nftftd. 

” *®  Intelligence  sources  used  In  preparation  of  the  erfsslon  daU.  wid 

u!^  ***  perception  of  the  nission  need  as 

^ Defense  decision  and  subsequently  as  the  basis  for  stating 

the  nisslon  need  In  soliciting  co^ietltlve.  Innovative  alternatives  as  potential  solutions. 

that  «r*f^»!Ihi4^4  ****  doc«ntation  that  will  require  a change  In  approach  Is 

progran  constraints.  In  addressing  the  problen  of  sollrttlng 
concept  « the  ^s  of  a statenent  of  nisslon  need,  the  respme  frt» 
relattonshlp  to  Investnent,  nanpower  denands.  operational 
Inpllcatlons,  the  1nvo1ve«nt  of  allied 

rir^4’»!^f;;  ^tralnu  are  established  as  the  result  of  a problen 

cycle  rather  than  before  the  fact  a»esswnts. 

It  Is  Intended  to  piece  essentiftl  constrelnts  In  full  force  it  the  stert  This  reoulre- 

appllcftble  to  ell  solutions  Is  en  inpomnt  pert 
“ “• •'  >*«"• 

— ...4I55  ®fj^®l«ng  dollar  resources  coninitted  to  solving  a specific  nisslon  need 

techniques.  Dur  treatnent  of  funding  needs  In  the  past  have  bee”^ 
to  ^dget  for  cos^  prepared  as  direct  estinates  of  a proposed  systen  well  before  the 

ultinately  ^e*ha2oc*Sltt‘'^«Hl'«  .^'^«"ce  s^ws  clearly  that  risk  and  laicertainty 

» ®r  constioints  on  the  Investment  to  be  made  by  Defense  In  procuring 

^™n  future  budgets.  The  statenent  of  funding 
Stli^l  ®"  ‘•'f  “®‘  esMmate  of  a projected  systen  and  n>re  liqwrt- 

iS^M  J The  ODD  works  with  5.  10  and  15  year  pioject- 

^ planning  process.  A portion  of  future  budgets  Is  always  comirftted 
Kthlf  0P«n  budget  available  to  meet  new  needs  Is  not  unlimited, 

within  this  constrained  budget  structure  the  coranltments  to  future  expenditures  for  new 
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*1*^  I?^^2  **?  ^***'*‘‘  ‘’y  t*>e  Govenur^nt  and  the  contractors* 

- Mat  ^“"‘'■actor  be  expected  to  make; 

, "J  iJ'SSSii;  EEsilS^'**"  “"’ '»'  ™ ■»«  «"«t- 

the  *“!?'’  ''*•  *"'*  t*!®  WD  Cbpponents  to  develop  ” 

Dlac^hn[^«I!l™^?^«I*“***'^  policy  and  procedural  techniques.  Implemitatlon  niBt  take 

Decentralization  ' .-.i  .:  ;.  ■■.■.•: 

w^fh  CMtrallzatlon  versus  decentralization  has  been  discussed  extensively 

^th  stwg  arguments  for  more  of  both.  Without  attempting  to^clde  Sriss^e  1^  ^ 

T <telegate  selected  program  reviews  to  the  Service  SecretaHes  at 

fenS”  H ll^^ized'^hfni?!  Sectary  and  recommendations  made  to  the  Secretary 

resSblilti^f^thTstcretarTof  Sfensir^""  declslonmakli; 

condua®the'^rei(J^?eI“’''ThI‘fOsw?’r'  ‘5®  (S)SARC,  has  been  established  to 

sors  similar  In  conposition  tl^the  Sc  S^SlSARC  ^'‘n 
^e  same  policies  apply  to^th^K^  llJludlJ^Se  u«  ^Jbe^InTsu^  r 
slcS?"of°tefSs?'  Secretary  -ho  will  in  turn  make  his  recooDendatlons  to  the 

regarding  decentralization  continue  to  be  split.  The  action  taken  to 

! JiriEX’Er.SS 'd  *SsES;.TKV:E,i!*oT,EE?’i 

ss"Essfis  s'EhS?sje“E».“E„'"E?:r‘.i.s”: 
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These  principal  changes  to  the  000  nanagenent  process  for  major  system  acquisition  ' 

are  extensive  and  the  adjustments  to  be  made  throug^t  the  broad  areas  of  the  functional 
and  operating  elements  idll  not  fall  Into  place  ifithout  in  effort.  To  make  changes  In  : 

deep-seated  operating  methods  In  any  large  organization  Is  a difficult  task  and  the  000  1 

should  expect  this  task  to  be  difficult.  The  basic  policy  changes  have  been  made.  000  ; 

Directives  5000. 1»  ”Kijor  System  Acquisitions.”  andSOOO.2.  "Major  System  /^quisltlon  * 

Process,”  mere  Issued  18  January  1977.  The  two  policy  Directives  implemented  the  provi-  1 

slons  of  A-109  and  were  effective  Immediately  wlctiin  the  000.  All  related  000  policies  j 

and  procedures  were  directed  to  be  changed  to  cosply  irlth  the  new  Directives.  ^ ! 

> 

Action  by  the  000  Components  to  Issue  Internal  policies  to  Implement  the  changes 
were  to  be  completed  In  Nay  1977;  however,  some  delays  In  meeting  this  schedule  have  ; 

been  encountered.  The  actions  by  the  Coi^nents  should  be  completed  without  excessive  \ 

del^s.  Notwithstanding  these  delays,  the  new  policies  are  in  effect  and  will  be  followed  ] 

in  the  initiation  of  new  system  prorrams.  This  is  not  to  say  that  there  is  instant  change  . 

in  the  process— Instant*  change  has  not  been  planned  nor  expected.  ! 


The  new  policies  will  not  be  back-fitted  to  on-geing  programs.  Pregrans  that  have 
progressed  to  various  stages  In  the  system  acquisition  cycle  will  not  back  track  over  the 
completed  actions  to  make  adjustments  required  by  the  new  policy.  A program,  for  example, 
that  has  progressed  beyond  DSARC  I,  (the  previous  reference  to  Miles  tone  I)  will  not  be 
required  to  comply  with  the  new  policy  covering  the  p-'se  activity  back  to  Milestone  0. 

The  new  policy  does,  however,  apply  to  all  subsequent  program  actions. 

The  transition  to  the  new  procedures  will  be  awkward  for  some  initial  period  In 
dealing  with  programs  at  various  points  In  the  process.  Certain  prograas  projected  by 
the  Services  as  new  starts  In  fiscal  year  1978  arc  not  In  fact  new  starts  under  the  new 
policy  since  these  programs  have  progressed  well  Into  the  concept  phase  under  the  prior 
policy.  The  Navy's  YSTOL  Type  A system  is  an  example  of  this  point.  The  program  was 
prepared  for  OSAKC  I at  the  time  the  new  policy  became  effective.  In  restructuring  the 
program  for  Milestone  0,  numerous  problems  were  encountered  since  the  program  had  progressed 
to  the  selection  of  a solution  concept  with  Secretary  of  Defense  approve!.  Generally, 
programs  approaching  DSARC  I under  previous  policies  have  been  underway  for  periods 
varying  from  months  to  years.  As  such  the  application  of  the  new  policy  must  be  accom- 
plished on  an  Individual  program  basis. 

A strong  argument  has  been  made  by  some  alleging  that  the  new  front-end  policy  will 
result  In  extending  the  system  acquisition  process.  This  argument  has  certain  appeal  on 
the  surface,  but  has  no  substance  In  the  light  of  more  detailed  debate. 

Concern  that  the  new  policy  for  Initiation  of  major  system  acquisition  programs  will 
extend  the  process  Is  based  on  the  addition  of  the  new  milestone  decision  point.  Milestone 
0.  The  position  Is  taken,  that  the  formal  process  will  be  extended  a rumber  of  months 
and  possibly  as  much  as  several  years  as  a result  of  the  requirement  to  submit  a MENS  for 
the  Milestone  0 decision.  In  alleging  that  the  system  acquisition  cycle  Is  extended,  the 
coirparlson  has  been  made  with  the  existing  DOD  process,  starting  with  DSARC  I as  the  point 
of  program  In  Initiation.  It  Is  true  that  In  this  comparison  of  the  two  procedures,  the 
new  process  Including  Milestone  0 as  the  point  of  program  Initiation  snows  the  formal 
process  to  be  extended  an  additional  phase.  This  coit)ar1son  Is  not  valid,  however,  since 
there  Is  no  recognition  given  to  the  conceptual  work  that  Is  completed  prior  to  DSARC  I 
under  previous  policy.  Past  experience  shows  the  conceptual  activity  to  be  prolonged  over 
several  years— the  B-1  program  is  an  exanple  where  the  conceptual  phase  extended  over  a 
number  of  years  prior  to  the  DSARC  I. 
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Fred  Dietrich 

Office  of  Federal  Procurenent  Policy 


I would  like  to  Ulk  a little  bit  about  sow  of  the  difficulties  involved  in 
transitioning  frow  research  to  policy  to  ieplewentation  in  the  area  of  Federal  Acquisition. 
Believe  it  or  not.  these  areas  are  inter-related  and  it  is  part  of  qy  job  to  see  that 
they  function  interdependently. 

Researcn  is  basically  the  develofwnt  of  a base,  a past,  present  and  advanced 
acquisition  technology.  The  product  is  a body  of  knowledge  about  aajor  system  acquisitions. 
Policy  is  a broad  conceptional  criteria,  not  detailed  procedures.  It  is  not  what  we  km 
as  the  Air  Force  375  series  of  procedures,  which  was  a policy.  Practice  is  the  iaple* 
menUtion  of  policy  at  the  operating  level.  Not  let's  try  to  interrelate  these.  In 
general  I look  at  it  as  a triangular  sitmation  with  the  three  apexes  of  the  triangle 
entitled  Policy.  Research  and  Body  of  Knowledge  and  Practice.  I put  the  policy  on  top 
of  that  triangle.  Of  course  Frank  Harden  over  here  put  Practice  on  the  top  and  Policy 
and  Research  on  the  bottom  and  Ted  Helner  from  the  Air  Force  Academy  certainly  put  Research 
on  the  top  and  Practice  and  Policy  on  the  bottom.  But  Tve  got  the  chalk  and  Tve  got 
the  podium  so  Policy  is  on  the  top  of  the  apex. 

Ideally,  we  would  have  a free  flow,  with  push  and  pull  in  each  direction  of  the 
three  legs  of  that  triangle.  Uhat  do  I mean  by  the  push  and  pull?  Take  the  push  between 
research  and  policy.  The  push  from  independent  research  is  pushing  policy  development.  . 

You  can  understand  how  that  might  happen,  and  I'll  elaborate.  It's  like  the  wet  noodle. 

The  pull  is  where  the  policy  makers  are  asking  for  research  results.  The  push  from  the 
policy  down  to  the  researchers  is  where  the  policy  makers  sponsor  research,  which  is 
the  ideal  situation  for  the  researchers.  And  the  pull  from  research  is  where  they 
independently  Initiate  research  based  on  policy.  This  push  and  pull  relationship  exists 
in  each  one  of  these  triangles,  or  legs  of  the  triangle.  Now  we'll  try  to  develop  them. 

Let's  examine  the  real  world  and  some  of  its  problems,  at  least  as  I perceive  them. 
I'll  state  right  now  that  I don't  have  pat  solutions,  but  I feel  that  if  we  sort  out, 
understand  and  discuss  the  problems,  solutions  are  possibly  less  difficult.  The  real 
world  we  live  in  consists  of  our  own  separate  worlds.  An  example  is  the  "policy  world? 

One  can  draw  a gate  or  a fence  around  it.  You  can  also  draw  a fence  around  the  world  of 
research  or  draw  a fence  around  the  world  of  the  practitioners.  Ue  talk  to  ourselves 
within  these  worlds.  It's  nuch  easier  for  me  to  comninicate  with  someone  who  understands 
me  so  I talk  to  other  "policy  people."  lie  have  a tendency  to  tell  ourselves  in  our  own 
little  world  how  great  we  are  and  coeplaia  abbut  the  other  worlds.  You  know,  they  don't 
understand,  they  don't  care  about  all  these  Important  things  that  I'm  doing,  or 
they're  not  trying  to  understand  ny  problems.  In  fact,  they're  not  even  listening.  The 
result  is  inevitable:  we  build  fences.  You  know,  ay  job  is  policy,  yours  is  research. 

For  exanple,  from  a researcher  here  is  the  typical  thing  that  we  mi^t  find:  "Here's  m/ 
research  paper  with  specific  recoonendation  for  reforming  the  acquisition  process.  Please 
send  me  a copy  of  the  policy  which  implements  qy  recomaendations.  By  the  way.  I've  sent 
copies  of  my  recommendation,  which  will  save  billions,  to  the  President,  the  Government 
Operations  Conmlttee,  the  Congress,  the  GAO  and  Comoon  Cause." 


586 


9 


A 


/ 


evoi  dcniKi  or  cnvUlo  oboJt'alSoier^i.  ““!»'«  '«  “at  bofore 

po1nts*tharrB*tr^I!^  to  ^ heto”*Th^iJ^ST“f  ***  understanding  the 

Industries  Cosparatlve  Inv»<tMnfe  im  fro®  the  Sloan  School  Study  of 

- The  push  Is  an  Industry  solution^ookinq'for  Tn^d  acquisition, 

thing.  That's  like  AWACS  A null  fKo  ^ understand  that  type  of 

like  when  all  the  requi^ts^Ji  wUbliJS  to  a government  need.  That's 

try  responds  to  the  need  The  studv  <locumented  and  then  indus- 

investo^t  on  the^rtlf  a con^Scto?^  to  «ver  three  times  the 

-ay  thnn-gh  the  acquit tion  oSSr^s  half 

-ords,  selling  Uie^d  as  ueiras*th^I^^rt.!r/iJ“* 

a product  to  an  approved  nert.'  ThifsayTth^Jt  lUiSdiU®!*  expewive  ttan  responding  with 
the  Story  is,  don't  ex^t  i^oend^ili.tr^.J^S®'*!*  is  real.  The  roral  of  . 

those  th^t  tn  reqisteT  '****”="  be  as  readily  accepted  as 

sasissj  roffc^As  r *“ 

nurtured  it,  and  effectively  marketed  it  researched  the  concept, 

ai^  by  Msisting  those  who  operltS^nd^r^it!  'Ni^^to’co$t^'‘l«f  Ihi®^ 

... «« «.  » ,o,,„  . ,0.., COO.M.C. 

experience,  research^^  nasrt^n^iwn^r  P"”***  ^he  current 

Acquisition.  T^m^pT^omc^L^  **“’’'*  '^conwndatlons  in  Major  Systems 

by  coordinating  with  all  the  96  aoer^ies  in  ^ ^ ” ? *''*  ^"'^eut  practices 

(about  30  of  them),  industry  toe^  !Ij  ’ Government,  industrial  associations 

fact,  one  of  our  J^eat  anta«ntoto^  £ In 

had  a significant  inout  ^chanLi  ncpp  ”<>“*«  Armed  Services  Connittee, 

participation  in  the  major  svstem^acoui«ifinI®n^  Sreater  cognizance  to  the  laboratory 

S'sH;  ?«  . 

before.  If  someone  doesn't  u^e^tand^erm^^ikS  -iiL*  **  «ccused  of  that 

they  call  it  “Pentag^se  * K findinT^  I t!  ownership.- 

up  to  the  fact  that^t  they  mIT  plntl^n!!!  It  * *?”*  a*^  -aking 

renected  in  son.  iSmnto^in  tJi  te^en^^ar  J!?’  termtodogy.  WB  circular  is 
««.w.«ni5  in  me  uerense  Department.  For  example,  there  are  5001,  5002, 
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OSD  level  documents.  In  the  Navy  there  are  OFNAV  Instructions » NAVMAT  Instructions* 

NAVSEA,  NAVAIR,  etc,  etc.  Instructions.  In  the  Air  Force  we  have  the  0MB  circular,  OSD 
and  then  they  respond  with  the  Headquarters  Air  Force  Reg.  and  AFSC  and  SAMPSO  on  dcMn  i 
the  line,  ASD  and  so  on.  In  addition  to  this  layering,  we  find  fragmenUtlon  of  A109. 

A109,  if  you  recall.  Is  signed  by  two  people;  High  Hit  and  Jim  Lynne.  The  reason  it  was 
signed  by  two  of  th^  is  b^ause  one  had  the  authority  that  the  other  one  didn't  and  vice 
versa.  Hit  had  the  procurement  regulatory  authority  which  in  fact  Is  reflected  in  A109 
and  Lynne  had  the  authority  to  tell  the  agencies  how  to  deal  with  Congress,  how  to  submit 
budget  information  and  how  to  manage.  That  was  in  his  charter.  It  was  not  in  Hugh  Hit's. 

So  as  a result  we  confined  all  these  things  into  one  directive.  However,  when  it  cones  i 

over  to  the  Defense  Department  we  find  that  they  are  going  to  make  it  5001,  5002  , 5003, 

5004  , 5030,  number  34  and  4105,  62  plus  others  and  the  ASPR  is  to  be  changed  to  implement 
the  policy.  Now,  until  we  get  to  the  practitioners  in  the  field,  what  happens?  Hhat 
do  we  find  when  we  get  out  and  talk  to  a practitioner?  The  main  thrust  is  lost.  There 
are  conflicting  Instructions,  multiple  Interpretations,  and  they  never  heard  of  A109  or 
the  reforms  it  requires.*  Government  people  are  frustrated  by  there  situations,  especially 
when  th^  face  a contractor  on  the  other  side  who  has  received  A109  directly;  read  it, 
put  on  internal  briefings,  and  understands  it.  The  Contractor  then  gets  frustruted  on 
the  other  side  of  the  table  and  wonders  if  the  Government  is  ever  going  to  change. 

Thus,  there  is  necessity  for  some  orientation  and  training,  at  all  levels.  IMl 
ask  in  here,  how  manyare  Interested  in  the  work  that  they  do?  May  1 have  a show  of 
hands?  Yeh,  great.  Now,  how  many  have  read  A109?  That's  a good  percentage.  1 think  I 
sand-bagged  you.  How  many  have  read  OFPP  circular,  pamphlet  no.  1 which  talks  about 
the  application  of  A109?  He  went  on  a thirteen  city  tour  and  grabbed  some  people  from  the 
Defense  Dep»rtrent,  Bob  Berry,  and  A1  Espisito.  He  had  NASA  and  ERDA  representatives  and 
representatives  from  Industry  who  had  some  experience  with  A109  and  held  conferences  in 
thirteen  cities.  We  got  a lot  of  feedback  from  those  conferences  and  learned  some 
interesting  things,  which  I'll  cover  in  a bit.  The  rerrer.entatlves  from  NASA  are  going 
to  brief  everybody  in  NASA  in  the  next  two  months.  They're  covering  the  twelve  centers 
with  some  nine  briefings.  They  have  already  published  some  changes  in  their  source 
selection  procedure  manual  and  they've  Issued  their  top  policy  docianent.  They've  also 
put  out  a paper  that  can  be  read  In  abcut  minutes  which  tells  of  the  change  and 

the  way  they're  ^Ing  to  do  business  as  a result  of  A109.  They're  moving  and  they 
undersUnd  that  It  is  serious.  Other  agencies  have  not  rewritten  A109  in  Impleioenttng 
It,  but  have  attached  /j09  to  their  iapiementing  Instructions.  In  a couple  of  cases, 
these  Implementing  Instructions  reference  OFPP  1,  which  talks  about  the  application.  In 
fact,  one  agency  has  an  Implementing  Instruction  that  is  all  on  one  page,  which  is 
certainly  refreshing.  This  Instruction  defines  a major  system  for  that  agency,  appoints 
an  acquisition  executive,  establishes  an  operating  MSA  review  board,  assigns  responsibili- 
ties for  program  management  appointments,  and  says  to  follow  A109  and  use  OFPP  1 as  a 
guide  in  implementation  of  A109. 

Now  let's  take  a look  at  another  gap-that  gap  between  the  practitioner  and  the 
researcher.  First,  we'll  look  at  the  pull  of  the  practitioner  and  the  push  by  the  research- 
er. For  example,  I'm  going  to  acquire  a new  prograra  manager  who  has  Just  cone  out  of 
the  cockpit,  or  off  the  bridge,  or  out  of  the  tank  and  I'm  going  to  acquire  a new  major 
system  following  A109.  I've  just  been  appointed  and  have  been  through  the  school  at  Fort 
Bel  voir  for  five  months  and  have  learned  how  we  did  major  systems  acquistlon  under  5001 
and  5002  dated  1971.  The  project  officer  may  not  have  the  personal  experience  necessary, 
but  it's  recognized  that  everyone  doesn't  have  It.  Therefore,  he's  likely  to  ask  the 
research  group  to  assist  him  in  the  planning  or  give  him  a dump  of  the  lessons  learned 
and  the  applicable  body  of  knowledge.  His  request  Is  usually  urgent,  and  unpredictable. 

It  certainly  would  disrupt  any  research  organization  that  Is  trying  to  do  things  in  an 
orderly  manner.  As  a result,  some  research  organizations  hire  a -Belt-Hay  Bandit.* 

588 


i 


] 


\ 


j 


I’ll  have  to  actilt  though,  that  I an 


..  5*  Bandit  Is  not  all  bad.  I'li  have  to  a(te1 

the  one  who  has  converted  the  President's  itanranHiim  

called  -Death  on  ConsulUnt^-  li  reJllJ 

President  or  myself.  It  basically  says  that  If^thlS^rnfit^  *!l5f either  the 
thing.  This  thing*  gets  the  awlslUon  er^r  1 1 ^ 5®  creates  sow- 

Ions  Involv^  In  It.  creative.  They  reinterpret  the  policy  and  the  dlrect- 

ril  just  take*i  s^^*«*arre*MDirof*lmrttf*  '•^***'".  **^  "O'"  ^ere,  but 

pretty  loud  and  clear  that  you  dTrot  oS  s«y» 

In  their  syste.  designed  coSpS.^flS  S2s?bS  • «»P^* 

features  from  each  one,  and  thm  reco»lt»n  ^ i**"’  ***'*  ***  •*«** 

systen  designed  concept.  All  the  AMrals  SU**  ®®’'em«nt  design  or 

follow  A109.  Rfqi  cane  out  of^  top  on  dam  said  they  were  going  to 

perforwd  and  the  r«^tlM  Inf^M^  th.t  had  a statenent  of  work  which  was  to  ^ 

the  streets  than  I got  thm  ^nc  calls*^cartS^f^*'''*K?»***4!!*’^' 

Fred,  we  don’t  want  to  plv  tte  ^ Thir7.^I!'*®1  " **»<»  1*  *109 

;«i>ond  to  this  and  It  san  rlghtTS  herJ  **4*®  2 "HHon  dollars  to 

They  want  all  our  Innovative  l^s  frSS^iS  t£'~  ? f*!!!  *“  I*  t"nsFused. 

Issue  us  a solicitation.-  "**”*  ™ ^^"0  *®  Flck  and  choose  and  then 

Ik*  £ iU'SL'S,"  f" “*•'  •« 

was  not  their  Intent  at  all.  ^ * ’«“«'•  "“t  said  that 

-You  <ton’I“aSrti%We^2l?ry2MnISJ!tWe^^  ^5^*  “I"*-'  *•«• 

I«oS.;g5!"w!toTJr^to™!„S  tototli,®Sn‘Se‘?art  of 

i'lT*'"  rsr’T'JkStr* 

It  night  be  better  to  modify  an  F14  to  should  be  doamented. 

w?f-  L’7sT“!“ 

K.;'£S&:n!‘HfrS 

you  their  brochures  a.o  vw  ntoacti!*  Ibnagenent  Inc.  and  1*11  tell 

what  the  researchers  tov^to  to.  -rketing  and  Krehandising.  And  that’s 
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Let's  look  at  the  pull  by  a researcher  and  a very  week  push  by  Vm  practitioner 
in  the  area  of  lessons  learned  frcm  A109  inpleventatlon.  There  Is  little  or  no  Motivation 
on  the  part  of  the  practitioner  to  docuMent  or  provide  lessons  learned  for  the  researcher 
so  that  he  can  expand  the  body  of  knowledge,  analyze  and  evaluate.  Therefore,  it's  up 
to  the  researcher  to  expand  that  body  of  knowledge  by  taking  the  initiative  to  gather  the 
data,  analyze  it,  evaluate,  draw  a conclusion,  stake  reccxMwndatlons,  and  Market  the  product.' 
To  Me,  this  is  essential  in  the  process;  to  build  that  body  of  knowledge;  to  disscMinate 
it,  and  to  advance  the  ManagcMent  technology  for  Major  systcMS  acquisitions. 

I have  a lot  of  lessons  learned  that  we  Might  cover  in  the  question  and  answer 
period,  but  in  suMMary,  lets  swing  the  gates  and  the  fences  both  ways.  Let  us  recognize 
that  each  world  in  the  universe  has  its  unique  problews  and  close  these  gaps  between  the 
researchers,  the  poliqr  Makers  and  the  practitioners. 
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The  Independence  of  the  United  States  can  be  aaintained  only  if 
^ *****  ^ establish  and  support  an  adequate  defense  force. 
Bxlst^  weapon  systeas  aust  be  operationnUy  ready  and  new  technology 
**  *®*^*”*  insure  that  the  operational  ailitary  units  can  cope 
with  ^ t^at  Croa  a potential  adversary,  other  national  priorltie^ 
^lute  a justifiable  ooaoem  for  the  health,  education  and  imlfare  of 
^ cltl«^.  ms  ■iltlpUcity  of  needs  aeans  that  the  Oeparteent  of 
receive  Halted  resources  with  which  to  effect  its  defense 

Tta  limitation  on  available  r-^sooroes  means  that  the  funds 

Por^s  aost  be  expended  in  an  efficient  and  effecUve  Banner. 
ThH  roquir^Mt  places  a premium  on  adequate  and  tlaely  planning.  Each 
i^lita^  aervlM  Bust  require  froa  each  subordinate  unit  an  acconnUng 
of  fun^  expen^  and  this  necessitates  the  comparison  of  results  agaLst 

'or  the  Introduction  of  the  new  technology 
. and  corollary  weapon  systeas  is  the  weapons  acquisition  process.  As  a 
consequ^e  of  the  limited  funding,  acquisition  and  procurement  planning 
becwae  <^tic^  to  each  phase  of  the  weapons  acquisition  process.  Four 
priaary  decision  points  are  now  associated  with  the  acquisition  of  a 
eupM  system.  Office  of  Hanageaent  and  Budget  Circular  Ho.  A-109  has 
tne  r^pMt  of  adding  a Defense  System  Acquisition  Review  Council  (DSARC) 
sero.  ms  decision  point  involves  the  idenUficatlon  and  definition  of 
a specific  mission  need.  DSARC  1 at  the  end  of  the  conceptual  phase 
around  the  selecUon  of  a oraapetltive  system  design  for  test. 

DS^  2 subsequent  to  the  validation  phase  involves  a cowitaent  to 
full-scale  development  and  limited  production.  The  most  significant 
tocision,  however,  is  the  DSARC  3 which  coasts  the  system  to  full  pro- 
ducHon.  For  major  systems}  that  is  one  with  an  estimated  production 
cost  exceeding  $300  oiillionp  a production  readiness  review  (PRR)  is 
reguired. 

The  Production  Readiness  Review  (prr)  is  an  Air  Force  procurement 
concept  that  evolved  in  the  early  1970s  to  meet  the  demands  of  the  dynamic 
^apons  acquisition  process.  A PRR  is  a government  analysis  of  a con- 
^tor  s readiness  to  transition  from  the  full-scale  development  phase 
roto  the  production  phase  of  the  weapons  acquisition  process.  The  con- 
tractor may  have  produced  some  developmental  heurthfare,  but  the  question 
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addressed  by  the  PRR  is  tfhether  or  not  the  contractor  is  ready  to  produce 
the  required  quantity  of  production  units  efficiently  and  economically. 

Air  Force  Systems  Command  (AFSC)  defines  the  Production  Readiness  Review 
(PRR)  concept  as  follows:  ^ 

The  PRR  is  a formal  inspection  to  determine  whether  (1)  a 
system  or  equipment  under  development  is  ready  for  efficient 
and  economical  quantity  production;  (2)  all  ia^rtant  pro- 
duction engineering  problems  encountered  during  dexelopaent  < 

have  been  resolved;  and  (3)  the  contractor  has  accomplished 
adequate  planning  for  the  production  phase  (4:1-2)  • . ^ 

^ - • ' . . * • j . ir,?  I 

The  purpose  of  this  paper  is  to  report  the  results  of  a study  to 
determine,  first,  if  significantly  different  PRR  approaches  have  been 
used  on  major  Air  Force  programs  to  date  and,  second,  if  a standard  PRR  ^ 
approach  is  feasible  for  conducting  future  PRR  programs. 

. BACKGROUND  ^ ‘ y 

. Cost  growth  and  large  cost  overruns  on  many  military  programs  in 
recent  years  have  generated  severe  cxltlcismm  in  the  press  and  in  Congress. 
While  inflation  and  an  expanding  technology  are  increasing  the  costs  of 
weapon  systems,  the  growing  demand  for  public  funds  to  support  other 
government  projects  has  resulted  in  a lesser  proportion  of  the  federal 
budget  available  for  Department  of  Defense  prograas.  • V..:-  I 

t ^ '-I  i*.  L ^ 

The  acquisition  of  major  weapon  systems  is  designed  to  follow  a 
systematic  flow  through  well-defined  phases.  These  phases  are  conceptual, 
validation,  full-scale  development,  production,  and  deployment  (see 
Figure  1)  • A major  step  wrs  taken  in  1969  to  achieve  better  coordination 
of  the  various  phases  of  the  weapons  acquisition  process.  The  Defense 
System  Acquisitiop  Review  Council  (DSARC)  was  formed  to  evaluate  and 
review  each  major  weapon  system  at  three  critical  junctures — DSARC  I 
before  transitioning  into  the  validation  phase,  DSARC  II  prior  to  inoving 
into  the  full-scale  development  phase,  and  DSARC  III  before  moving  into 
the  production  phase  (see  Figtire  1) . 


The  purpose  of  DSARC  III  is  to  provide  a recooimendation  to  the 
Secretary  of  Defense  on  moving  a major  weapon  system  into  production. 

The  DSARC  III  must  confirm,  among  other  points,  "...that  a practical 
engineering  design,  with  adequate  consideration  of  production  and 
logistics  problems,  is  complete.”  DSARC  III  serves  as  the  basis  for  the 
decision  on  whether  or  not  a weapon  system  will  be  produced  for  deploy- 
ment. DSARC  III  meetings  are  held  when  the  military  department  determines 
that  engineering  and  operational  systems  development  and  testing  have 
been  substantially  castle  ted,  all  major  development  problems  have  been 
resolved,  and  the  weapon  system  is  ready  to  transition  into  production. 

For  some  of  the  newer  major  weapon  systems  (e.g.,  the  F-16  program),  a 
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I^SARC  II lA  is  held  prior  to  receiving  long  lead  production  authorization 
and  a DSARC  II Ib  is  held  before  full  production  go-ahead  is  received. 

The  Host  critical  tine  period  in  tens  of  the  rate  of  expenditure 
of  funds  is  the  production  phase  of  the  wapons  acquisition  process  (see 
Figure  2).  The  technical  uncertainty  level  should  have  been  reduced  by 
this  tine,  and  the  najor  obstacle  is  to  convince  DSAKC  and  Congress  that 
the  program  is  ready  to  transition  from  the  full-scale  development  into 
production.  One  estimate  of  a typical  Air  Force  program* s costs#  excluding 
operation  and  s«q>port  costs#  is  that  about  33  percent  of  a program's  funds 
<UDe  committed  for  research  and  development#  and  the  other  67  percent  are 
spent  in  the  production  phase.  The  average  amount  of  Air  Force  Systems 
Command  production  dollars  spent  in  fiscal  years  1971#  1972#  and  1973  was 
1.7  times  greater  than  the  expenditure  for  RDT&E  activities. 

In  1970#  Mr.  Philip  N.  Whittaker#  the  Assistant  Secretary  of  the  Air 
Force  for  Installations  and  Logistics#  expressed  concern  about  whether 
sufficient  eiq>hasis  was  being  given  within  the  Air  Force  to  the  preparation 
for  and  the  management  of  the  production  f^ase  of  major  weapon  acquisitions. 
An  Air  Force  study  was  conducted  to  determine  if  the  Air  Force  had  accept- 
able procedures  to  (1)  accoeplish  production  planning  daring  the  full-scale 
development  phase  of  the  weapons  acquisition  process#  (2)  formally  docuwnt 
and  review  production  criteria  prior  to  the  production  go-ahead  decision# 
and  (3)  continuously  monitor  the  production  program  after  the  production 
decision  is  sukde.  The  study  team  conducted  cosqprehensive  reviews  of  Air 
Force  Systems  Command's  production  manaigement  activities  in  various  field 
organizations#  system  program  offices#  and  contractor  plants.  One  specific 
recoamendation  was  that  the  Air  Force  should  conduct  and  document  a formal 
Production  Readiness  Review  (PRR)  prior  to  the  production  decision. 

The  Air  Force  stiidy  group  noticed  an  information  gap  when  it  came 
time  to  determine  if  a contractor  was  ready  to  transition  from  the  develop- 
ment to  production.  At  this  stage  many  of  the  major  %ieapon  system  relation- 
ships may  be  categorized  as  that  of  bilateral  monopoly.  The  monopsonist 
is  the  government#  and  the  monopolist  is  the  sole-source  contractor.  The 
Air  Force  initiated  the  formal  PRR  procedure  to  obtain  information  for 
evaluating  a sole-source  contractor's  readiness  to  make  the  development- 
to-production  transition.  An  AFSC  regulation  (AFSCR  84-2)  was  published 
to  cover  application  of  the  PRR  concept. 

Air  Force  Systems  Comnand  Regulation  (AFSCR)  84-2  prescribes  that  the 
PRR  team  size  and  cooq>osition  be  established  based  on  the  scope  of  the 
review  effort.  Primary  PRR  team  members  normally  possess  e}q>ertise  in 
the  following  functional  areas:  (1)  industrial  or  production  engineering# 
(2)  production  management#  (3)  program  or  project  planning  and  production 
control#  and  (4)  manufacturing  methods#  tool  design  and  test  requirements# 
plant  layout#  etc.  Since  the  procurement  process  involves  many  related 
functional  disciplines#  the  PRR  team  may  need  to  be  augmented  by  experts 
from  other  functional  2u:eas. 
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►t«n  84-2  also  states  that  the  PRR  team  conducts  an  on-site  exaaina- 

working  papers,  documents,  production  methods, 
processes,  and  techniques.  To  assess  the  contractor’s  readiness  to  tran- 
Th*  ODD  " ® ***  uction,  the  PRR  team  examines  existing  data  and  procedures. 
The  PRR  team  uses  predetermined  criteria  as  standards  to  conduc/an  orderly 

review.  WSCR  84-2  contains  twenty-five  evalJI- 
PigCr^Jf!*^^  questions  to  be  used  in  conducting  PRR  programs  (see 

» , f 

a M , 

84-2  further  stipulates  that  if  a contractor  does  not  meet 
acceptable  standards,  then  corrective  action  within  the  scope  of  the 

weak  areas  are  identified, 

‘ subsequent  reviews  is  developed.  At  the  conclusion 

of  the  PRR,  the  team  director  submits  a formal  report  to  the  Air  Fc-ce 
Program  Manager.  The  report  describes  the  contractor’s  weak  areas  Ind  * 
g ves  a recoomendation  on  the  program’s  production  readiness  status, 
rog^  Managers  and  other  Air  Force  management  levels  use  the  final  PRR 

^ productiOT  recomnendation  to  other  Department 

of  Defense  agencies. 

" I ? 

‘ Air  Force  Systems  Command  (AFSC)  has  three  major  product  divisions' — 
Electronic  Systems  Division  (2SD) , Hanscom  AFB,  Massachusetts;  Aeronautical 
Systems  Division  (ASD) , Hright-Pattersoa  AFB,  Ohio;  and  Space  and  Missile 
Systems  Organization  (SAMSO) , Los  Angeles,  California.  ProducUon  Readi- 
ness Renews  have  been  conducted  by  each  of  these  three  divisions  in  some 
form.  The  PRRs  which  have  been  coi^leted  or  are  in  progress  are  shown  - 


in  Figure  3. 
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RESEARCH  DESIGH  ASD  DATA  ANALYSIS 
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The  research  effort  had  two  objectives.  The  first  objective  was  to 
analyze  the  PTO  programs  that  had  been  completed  on  major  Air  Force  weapon 
system  programs  to  date  to  determine  what  disparities,  if  any,  existed 
among  the  PRR  approaches.  The  second  objective  was  to  determine  if  a 

stan^rd  PRR  approach  was  feasible  to  satisfy  Air  Force  Systems  Command 
requirenents. 

« « * • 

Since  Production  Readiness  Reviews  (PRR)  are  unique  to  Air  Force 
Systems  Command  (AFSC)  procurement  activities,  the  universe  for  the 
tese^ch  consisted  of  those  major  weapon  system  programs  that  were  reouired 
to  adhere  to  AFSCR  84-2  and  which  had  an  estimated  Research,  Development. 
Test  and  Evaluation  (RDTiE)  cost  exceeding  $75  million  or  an  estimated 
production  cost  exceeding  $300  million.  Two  populations  .^ere  identified 
within  the  universe.  PopulaUon  I consisted  of. those  major  AFSC  weapon 
system  program  that  had  completed  an  entire  PRR  program  and  had  met  a 
DSARC  III  review.  Population  II  consisted  of  those  AFSC  major  weapon 
system  programs  that  are  presently  conducting  PRR  programs  or  will* be 
required  to  conduct  PRR  programs  in  the  future. 
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A census  of  Population  I was  conducted.  Population  1 consisted  of 
three  najor  weapon  system  programs:  (1)  Airborne  Warning  and  Control 
System  (AWACS)«  which  is  being  procured  by  APSC's  Electronic  Systems  Divi- 
sion, Hanscoa  AFB,  .Massachusetts;  (2)  F-15  System,  tdiich  is  being  procured 
by  AFSC's  Aeronautical  Systems  Division,  Wright-Patterson  APB,  Ohio;  and 
(3)  A-10  System,  which  is  being  procured  by  APSC’s  Aeronautical  Systems 
Wright— Patterson  APB,  Ohio.  The  three  major  weapon  system 
programs  have  gone  through  the  entire  PRR  process.  Data  collected  on 
PRR  program  included:  the  nund>er  of  PRR  visits  to  prime  contractor,  PRR 
visits  to  subcontractors,  different  skills  (System  Program  Office) , PRR 
team  members  (SPO),  PRR  team  members  (Air  Force  Plant  Representative 
Office) , different  skills  (APPRO) , supplemental  PRR  team  members,  duration 
of  PRR  program,  verbatim  APSCR  84-2  PRR  questions,  additional  PRR  questions, 
estimated  prr  program  cost  (SPO) , and  estimated  PRR  program  cost  (APPRO) . 

The  data  were  used  to  objectively  assess  if  differences  in  the 

conduct  of  the  three  major  PRR  programs  cospleted  to  date. 

The  authors  were  able  to  statistically  show  that  significant  PRR 
program  variations  existed  for  the  F-15,  A-10,  and  AWACS  programs.  Some 
notable  exaiq>les  include  the  following:  (1)  the  AHACS  program  office 
made  three  major  visits  to  the  prime  contractor  to  conduct  its  PRR  program, 
whereas  the  P-15  and  A-10  program  offices  made  over  twenty  visits  to  their 
respective  prime  contractors  solely  to  conduct  PRR  program  m'tivltles; 

(2)  the  P-15  program  office  made  2g>proximately  fifty  visits  to  subcontractors 
to  conduct  PRR  program  activities,  while  the  A-10  and  AWACS  program  offices 
held  considerably  fewer  subcontractor  prr  meetings;  and  (3)  the  P-15  prr 
program  was  conducted  in  five  months,  vrhereas  the  A-10  and  AHACS  PRR 
programs  lasted  thirty  and  twenty-nine  months,  respectively. 

APSCR  84-2  contains  twenty-five  PRR  criteria  :diich  are  to  be  used  to 
assess  whether  or  not  a major  weapem  system  is  ready  to  maira.  the  inrrlnrmwnt 
to-producUon  transiUon.  TO  determine  if  a standard  PRR  approach  is 
feasible,  ten  Air  Force  Systems  Cowand  (APSC)  production  iMnageiiient 
e;q>erts  were  reque.^ted  to  prioritize  the  twenty-five  APSCR  84-2  questions. 

The  APSC  production  management  experts  were  required  to  possess  the  follow- 
ing qualifications:  (1)  active  in  the  prcinction  management  area  for  at 
least  three  years,  (2)  Involved  in  the  nuuugeaent  of  major  Force 
weapon  systems  for  at  least  five  years,  and  (3)  involved  in  the  planning 
or  implementation  of  at  least  one  APSC  PRR  program.  The  key  determinant 
for  selecting  the  production  management  experts  was  their  previous  PRR 
experience . 

APSC  experts  agreed  on  the  relative  ijiq>ortance  of  evalu- 
ating the  production  management  areas  covered  in  the  twenty-tive  APSCR 
84-2  PRR  questions,  then  this  would  provide  support  for  the  hypothesis 
that  a standard  approach  is  feasible  for  the  conduct  of  future  Production 
Readiness  Review  (PRR)  programs.  The  ten  AFSC  production  management 
e;q>ezts  were  requested  to  independently  rank  order  the  twenty-five 
APSCR  84-2  PRR  questions.  Each  expert  sissigned  a “one"  to  the  PRR 
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qiwstion  that  was  considered  most  Important  and  a •twenty-five"  to  the 
AFSCR  84-2  PRR  question  of  least  importance.  The  primary  nonparametric 
statistical  test  used  for  evaluating  the  agreement  among  ten  AFSC 
e)qE>ert8  for  the  twenty-five  AFSCa  84-2  PRR  questions  was  the  Friedman 
two-wy  analysis-of-variance-by-ranks  test.  To  provide  additional  support 
for  the  results  obtained  from  the  Friedman  test,  two  additional  statistical 
i^M,**?**  P«rforaed.  The  second  test  was  the  nocqaaraaetric  Kendall 
Ooefficleat  of  Concordance  W,  and  the  third  statistical  test  used  for 
wrUication  purposes  was  tha  parametric  F test  using  the  Air  Force 
InsUtttte  of  Technology's  Omni  tab  II  computer  program. 

Both  nonparametric  statistical  tests-the  Friedman  two-way  analysis- 
of^i*nce-by-ranks  test  and  the  Kendall  Coefficient  of  Concordance 
SM  the  parametric  F test  in  the  one-way  analysis  of  variance  supported 
the  same  omlusion  that  the  ten  AFSC  production  management  e»>erts  did  1* 
agree  on  the  ^Ucatir>n  and  inq)ortance  of  the  twenty-five  AFSCR  84-2  PRR 
questions.  The  prioritized  list  of  PRR  questions  from  the  most  iinortant 
to  least  Important  is  provided  in  Figure  4. 

Several  corollary  findings  resulted  from  the  research.  These  findings 

^vlew  of  Air  Force  publications  pertaining 
to  PRR,  discussions  with  production  and  procurement  personnel  who  have 
toen  acti^y  tovolveU  with  PRR  activity,  and  the  researchers'  interpre- 
tations of  their  studies  and  observations. 

attest  was  made  to  conduct  an  initial  survey  concerning 
toe  following  question:  When  should  the  PRR  program  acUvity  actually  . 
begin  in  the  weapons  acquisition  process?  The  ten  AFSC  production  manage- 
requested  to  indicate  prior  to  whidi  SSARC's  (i.e., 

OSARC  I,  H,  and/or  HI)  should  toe  twenty-five  PRR  questions  be  evaluated. 
The  ten  AFSC  experts  agreed  that  all  twenty-five  PRR  questions  should  be 
evaluated  prior  to  CSARC  III.  And  five  or  more  of  toe  ten  AFSC  production 
management  experts  agreed  that  twenty  of  the  twenty-five  AFSCR  84-2  PRR 
questions  should  be  evaluated  prior  to  rSARC  II.  The  consensus  of  the 
Production  management  experts  was  that  initial  PRR  activiUes 
Se“^  Quoted  much  earlier  than  just  prior  to  toe  preparations  for 
the  OSARC  HI  decisioD-fflikxng  process. 

Second,  toe  general  consensus  of  the  personnel  Interviewed  was  that 
there  is  a lack  of  readily  available  documentation  on  how  previous  PRR 
cOTductei  to  date,  so  readily  accessible  lessons- 

programs  that  are  now  required 

^ ^SCR  8^2  to  conduct  PRR  programs.  Historically,  it  has  been  incum- 
tent  M each  system  program  office  to  determine  what  previous  prr  programs 
had  conducted,  contact  personnel  who  had  been  involved  in  those 
previ^  ra.R  programs  to  determine  what  had  actually  been  accompUshed, 

Md  finally  develop  a new  PRR  program  plan,  using  their  researrii  informa- 
tion as  a baseline,  that  would  best  fit  their  particular  program. 
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Figure  4 
« . ^ • 

AGGREGATE  PRIORITY  LIST  OP  TWENTY-PIVB 
, AFSCR  84-2  PRR  gOESTIONS  PROM  TEN  APSC 
*X  - PRODOCTIOM  MANAGEMENT  EXPERTS 
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Current 
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APSCR  84-2  P51R  Question 
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I 

i 


2 


3 


• m.  ^ . 

« * . 1 . advanced  production  planning  has  been 

- . » accomplished  and  required  production  controls 
* established  to  ensure  timely  production* 

, ^ ^^i^eerinq  problems  encountered  during  development 
have  been  resolved  with  appropriate  trade-offs 
-w  - . ^ egeinst  stated  operating  requirements  so  that  ^ 

V;  " productiom  coets/scbedules  are  optimized. 

1 Hilestaaes  which  demonstrate  the  achievement  of  a 
^ practical  and  producible  engineering  design  have 
■ been  met. 

Results  of  technical  reviews  and  the  production 
iflg>act  of  unresolved  problems  and  risk  have  been 
assessed. 


13 


14 


19 


System  configuration  has  been  reviewed  to  determine 
if  any  significant  design  changes  will  be  required 
for  manufacturing. 

Test  program  results  and  the  status  of  qualification 
testing  to  determine  production  impact  and  risk 
have  been  evaluated. 

Production  or  manufacturing  c«q>abilities  of  major 
subcontractors  and  vendors  have  been  te<dinically 
evaluated  and  found  adequate. 

Assurance  of  readiness  of  the  manufacturing  and 
producUon  equipment,  methods,  facilities,  test  and 
training  equipment,  and  status  of  accessory  and 
ancillary  items. 
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Ranking 


Current 

Question 

Number 


AFSCR  84-2  PRR  Question 


10 

11 


12 

< - Zj  X: 
T*:  V 

13 


u 3 , Critical  production  engineering  and  production 
tooling  have  been  deoionstrated  to  prove  that 
engineering  has  been  satisfactorily  acco2»lishei« 

c ^ ^ 

4*  Acquisition  will  smoothly  transition  from  foll- 
r * ‘ scale  development  to  production. 

10*  Planned  production  schedules  reflect  economy  of 
. , operations  and  mininize  financial  coamitments 

r.14  c until  all  major  development  problems  have  been 
-cir?-’  resolved.  . • 

12  Change  acUvity  during  developi&ent  has  been  evalu- 
ated and  the  lB?>act  of  outstanding  changes  on 
production  has  been  assessed.  i; 

Specxfications  and  drawings  have  been  reviewed  to 
assure  their  adequacy  for  the  plcumed  production 
phase . 


15 

' “ j . . 


14 


15 


16 


17 


11 


21 


25 


16 


A thorough  assessment  of  the  make-or-buy  structure 
has  been  accoo^lished  and  procedures  exist  so  control 
and  visibility  of  the  vendors  and  suLcon tractors 
can  be  effectively  maintained. 

Quality  controls  and  inspection  procedures  have 
been  established  for  materials  treatment  or  pro- 
cesses to  be  used  in  production. 

Planning  has  been  made  to  assure  timely  release  of 
manufacturing  instructions. 

Application  of  production  tooling  and  test  equipment 
to  manufacturing  during  development  has  been  assessed 
and  the  application  of  same  to  the  production  phase 
has  been  defined. 


^Indicates  tied  pair  of  mean  rankings. 


601 


RAiiking 

Current 

Question 

Number 

AFSCR  84-2  PRR  Question 

* . " •*  ' 

18 

4 

18 

• ♦ •■'  2 

Production  aanegenent  systeas  used  for  providing 
aanageaent  with  tiaely  production  status  inforaation 
are  effective. 

19 

# 

17 

^ i ’ 

1 • 

Material  aanaoeKtit  systea  for  detexaination  of 
requireaents,  procuzeaent,  receiving,  inspection, 
aaterials  handling  and  storage,  inventory  control, 
control  of  finished  goods,  and  shipaent  is  adequate. 

20 

• •^  • V. 

7 

_■  ;'"■••«  * 

h systeaatic  approach  to  standardization  has  been 
accoaplished  in  the  design  process  and  parts  selec- 
tion to  aaxiaize  the  use  of  military  standard 
ooapooentSp  parts,  and  processes  consistent  with 
the  system  requireaents . 

21 

20** 

Constraints  of  laboratory  or  model  shop  capabilities 
versus  quantity  production  requirements  have  been 
fully  considered. 

22 

22** 

Assessment  of  the  GFP  or  services  requirements, 
controls,  management,  and  availability  of  siq>pliers 
has  been  accosplished. 

23 

24 

The  contractor  is  adequately  organized  to  acconplish 
the  production  requirements. 

24 

23 

Availability  of  production  labor  skill  requirements 
has  been  assessed  and  their  acquisition  aidequately 
planned. 

25 

8 

Product  assurance  controls  and  tests  to  prevent 
manufacturing  degiadation  of  perforaianoe  parameters 
have  been  established. 

**In(licates  tied  pair  of  nean  rankisgs. 
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AFSCR  84‘-‘2  is  directed  at  najor  weapon  systexn  prograas^  as 
defined  by  DOD  Directive  500.1.  The  AFSCR  84-2  does  not  provide  cuch 
assistance  for  the  saaller  Air  Force  Systems  Command  programs  which  are 
just  under  the  threshold  levels.  But,  personnel  for  the  smaller  programs 
ve  still  tasked  to  make  an  assessment  about  the  contractor's  readiness 
to  transition  from  full-scale  development  to  productiom.  Several  inter-  j 
view  respondents  expressed  the  opinion  that  a mini-PRR  program  should  - 
be  conducted  on  the  smaller  programs.  ^ I 

And  fourth,  information  gleaned  from  a background  review  indicated 
that  the  Air  Force  is  the  only  military  departamnt  that  conducts  a PRR 
prior  to  movement  into  production.  The  other  military  departments  conduct 
continuous  surveys  and  analyses,  but  no  formalised  procedure  has  been 
developed  to  acoosplish  the  task.  Additional  interviews  conducted  after 
the  initial  background  review  produced  similar  results.  Based  on  the  > 
authors*  research  findings,  the  Air  Force  remains  as  the  only  militaxy 
department  which  has  a formal  PRR  procedure.  Since  movement  into  produc- 
tion is  very  criticsl'and  expensive,  the  other  military  departments  should 
consider  establishing  similar  procedures.  * 

CXBCUJSIOIfS  AB  RBOONMENDATIONS  - r* 

The  PRR  approaches  used  to  date  have  been  different.  The  specific 
reasons  for  using  the  different  jqipzoaches  could  not  be  determined.  But, 
the  conclusion  is  that  it  should  not  be  necessary  to  start  at  the  beginning 
for  every  aspect  of  a new  PRR  program.  At  least  some  factors  or  activities 
should  remain  essentially  the  same  for  future  PRR  programs. 

The  data  oollectioo  and  analysis  indicated  that  the  ten  AFSC  produc- 
tion management  ejqperts  did  agree  on  the  relative  importanoe  of  the  twenty- 
five  PRR  questions.  This  conclusion  provided  SY]pport  for  the  hypothesis 
that  a standard  PRR  approach  is  feasible  for  application. 

The  use  of  the  PRR  is  relatively  new,  and  ai^lication  of  the  technique 
is  primarily  concentrated  on  the  major  Air  Force  weapon  system  programs. 
Although  only  three  Air  Force  PRR  programs  have  been  completed  to  date, 
many  more  Air  Force  program  offices  will  be  required  to  conduct  PRR 
programs.  For  :^xaBq>le,  the  F-16  System  Program  Office  is  presently 
conducting  its  PRR  program. 


t 

•f 

'*  J 


The  authors  recommend  that  future  PRR  programs  be  devel<^d  around 
the  prioritized  list  of  twenty-five  AFSCR  84-2  PRR  qxiestions  (refer  to 
Figure  4)  • The  questions  at  the  top  of  the  prioritized  list  should  receive 
the  most  enq>hasis  in  future  PRR  programs  i the  lower-ranking  questions 
should  receive  lesser  eaphasis.  Future  PRR  program  organizers  should 
acknowledge  the  critical  importance  of  accomplishing  the  top  six  qvmstions.  [ 

The  future  PRR  program  planners  should  recognize  that  a considerable  amount  1 

of  PRR  program  effort  should  be  spent  on  insuring  that  most  engineering 
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and  tesUiig  activities  have  been  accomplished  prior  to  the  production 

process.  The  contractor  can  be  initially  approadied 
priority  questions.  The  «*aining  nineteen  questi^ 

CM  be  br^n  into  sections  as  determined  by  tJie  individual  system  program 
offt^.  The  researchers  acknowledge  that  some  new  prr  evaluation  questions 
may  be  generated.  These  new  questions  can  be  placed  in  the  baseli^ 
prioritised  list  as  determined  ^ropriate  for  the  individual  PRR  program. 

Another  specific  recommendation  is  that  initial  PRR  planning  and 
ev^uation  activities  should  begin  as  early  as  DSARC  II.  two  of  the 
major  weapon  system  program  offices  which  have  coi^>leted  prrs  to  date 
spent  a^rorlmately  thirty  months  on  the  PRR  program.  This  recommendation 

4 «!****  that  the  PRR  program  should  actually  begin  early 

in  ^ weapons  acquisition  process  rather  than  just  prior  to  DSARC  III 
app^ch  is  recommended  in  which  the  entire  PRR  program 
effort  in  stages  that  will  coincide  with  other  prcg^  acti- 

vities. This  incremental  ^road>  has  proven  successful  on  PRR  programs 


which  have  been  conducted  to  date. 


-•  t.  ’•  . •'^fX 


1.  The  system  program  offi'ce  (SPO)  should  insure  that  adequate 
contrail  coverage  is  secured  for  all  contractors  (i.e.,  prime  contractor 
and  subcontractors)  Involved  in  the  PRR  program. 


2.  A PRR  preproduction  meeting  should  be  held  early  enough  in  the 
PRR  program  to  bring  all  the  participants  together  to  discuss  the  PRR 
pi^M  plan  and  the  specific  iig>leiBentatlon  of  the  plan.  The  convention 
should  prevent  many  potential  problems  downstream. 


3.  The  PRR  program  should  be  tailored  to  the  individual  program 
chuacterisUcs  using  the  prioritized  list  of  AFSCR  84-2  PRR  questions 
and  the  other  specific  recommendations  above  as  a baseline  model. 


*.t  H subcontractor  PRRs  are  conducted,  a prime  contractor  representa- 

tive should  be  in  attendance  at  each  to  prevent  the  govemnmnt  from  inad- 
vertently assuming  a position  between  the  prime  contractor  and  his 
subcontractors . 


5.  When  the  PRR  team  visits  a contractor,  the  team  leader  should 
make  explicit  remarks  in  the  initial  and  exit  briefings  that  the  con- 
tractor should  assume  no  contractual  direction  from  the  comments  or 
actions  of  any  of  the  PRR  team  members.  All  direction  should  be  given 
by  the  Procuring  Contracting  Officer  to  the  prime  contractor. 

6.  The  SPO  and  contract  management  organizations  should  begin  prr 
planning  early  in  full-scale  development  in  order  to  complete  the  PRR 
effort  prior  to  DSARC  III.  Sufficient  time  should  be  allowed  for  schedule 
slippages,  contractor  revisits,  and  Department  of  Defense  Product  Engin- 
eering Services  Office  (PESO)  visits. 
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7,  The  Air  Force  Program  Manager  should  direct  all  SPO  personnel 
to  coorider  their  support  to  the  PRR  effort  as  being  a very  high  priority 

; 'v  .1*  'c  i 


- • . • •.  f • •• 

6.  The  SPO  should  utilize  experienced  personnel  (e.g.,  Air  Force 
Materials  laboratory  (AFML),  previous  PRR  program  experts,  etc.)  to  . 

conduct  a oo^prehensive  PRR  of  the  ueapoo  system  program.  


r 

9«  The  PRR  team  should  be  specifically  tailored  to  evaluate  the 
particular  contractors,  equipment,  manufacturing  processes,  etc.  under 
review. 


. 10*  The  PRR  subject  should  be  addressed  early  enough  in  the  weapons 
acquisition  process  to  insure  that  producibility  and  production  readiness  . 
considerations  are  included  in  engineering  preliminary  design  reviews 
and  critical  design  reviews.  ^ ^ ^ ^ 

U.  ^A  systematic  approach  should  be  developed  to  handle  the  admixiis- 
trative  %iorkload  caused  in  managing  a PRR  program. 


12m  Gowemment  communication  with  subcontractors  concerning  PRR 
program  activities  should  be  conducted  through  the  prime  contractor  and 
not  directly  to  the  subcontractors. 


13.  The  SPO* 8 routine  communication  with  the  secocxlary  delegated 
contract  management  organizations  should  be  conducted  through  the  prime 
contract  management  organization.  As  a minimum,  the  SPO  should  keep  the 
prime  contract  management  office  informed  of  the  SPO  and  secondary  contract 
management  office's  activities. 


14.  The  SPO  should  document  PRR  program  activities.  Background 
documentation  that  is  used  as  a basis  for  the  final  report  to  the  Air 
Force  Program  Manager  should  be  prepared  to  provide  backup  justification 
for  the  recommendations  in  the  final  PRR  report. 


IS*  The  SPO  and  APPRO  should  determine  the  availability  of  existing 
data  sources  and  utilize  the  available  data  to  support  PRR  activities 
(e.g.,  pre-award  surveys,  AFCMD  Contractor  Management  System  Evaluation 
Program,  should ’cost,  source  selection  data,  etc.)* 


As  of  the  writing  of  this  study  report,  the  Air  Force  is  the  only 
service  to  have  developed  a formalized  PRR  procedure  to  2issess  a contractor's 
readiness  to  m^Lke  the  transition  from  full-scale  development  to  production. 
The  experience  on  the  three  Air  Force  programs — F-15,  A-10,  and  AHACS — 
that  have  co^leted  PRR  programs  has  been  encouraging.  All  three  major 
weapon  system  programs  received  favorable  DSARC  III  decisions  and  were 
authorized  to  proceed  into  the  production  phase.  The  interview  respondents 
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were  unaniaous  in  agreeing  that  the  PRR  technique  is  beneficial  and  should 
help  prevent  many  potential  problems  ir.  the  production  phase  of  the 
weapons  acquisition  process.  Due  to  the  critical  nature  of  the  development- 
to— production  transition  process  and  the  large  amount  of  tascpayers* 

whirfi  will  be  obligated  %#hen  production  go-ahead  is  authorized, 
it  is  recommended  that  the  other  military  services  develop  similar  Produc- 
tion Readiness  Review  (PRR)  procedures  to  insure  that  a contractor  is 
ready  to  go  into  production. 
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OONCUDZIIG  OBSERVATIOOS 


• i.V‘.  » 


After  analyzing  tbe  Air  Force's  PRS  technique,  the  authors 
that  the  PRR  technique  has  aerit.  First,  the  Air  Force  can  be  aoze 
assured  that  planned  major  weapon  systems  will  receive  a positive  DSARC  III 
and  Secretary  of  Defense  approval  on  full  production  go-ahead.  Three 
***3”  Force  weapon  system  programs— A-IO,  F-15,  and  ANACS — which  have 
collated  a PRR  program  have  received  production  authorizations.  Secx>nd, 
after  coa^leting  a PRR  program  effort,  the  Air  Force  buying  activi^ 
can  feel  aoze  confident  that  a contractor  is  ready  to  make  the  developnent- 
to-productioo  transition.  Tbe  iapact  im  that  many  potentiaa  problem  areas 
will  be  corrected  early  in  the  weapons  moquisition  process,  and  the  possi- 
bility of  a major  weapon  system  procurement  fiasco  will  be  reduced.  And 
there  will  be  more  Air  Force  and  contractor  interaction  and 
involvement  early  to  guard  against  potential  problems  that  could  be 
encountered  in  the  production  i^iase  of  the  weapons  acquisition  process. 

The  United  States  tazpayers  will  be  tbe  primary  beneficiaries  of  the  PRR 
program  activities. 
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Planning-  ud  "High-Low  Mix"  have  recently  be- 
highly  Inportant  to  the  Departownt  of  Defense.  This  paper 
wlU  present  a Method  of  analyzing  High-Low  Mixes  (and  relat^ 
procnrenent  strategies)  la  a context  of  Versatility  Planning. 

? **^*^*’  Versatility  Planning  and  the  lack  of 

■ethods  to  Inpleaenr  it.  Ve  will  also  consider  the  "High-Low  Mix" 

M an  ex^le  of  sonethiog  we  call  "Mixed  ProcureMnt."  which  Is 

*®  Parfotn  a given  nlsslon. 

w d^natrate  that  constrained  optlalsatlon  under 

for^wili  procurement  solution.  Ue  there- 

fo^  v^Iiirtn^*  *“'  ■**^‘*®'‘*  •'f*  “®‘  aultable  for  planning 

??f  *'*  adeguate  to  analyze  nixed  procurements. 

5^tion  3 will  propose  a solution  to  both  probless.  We  will 
portfolio  theory  to  purpose  a method  of  selec- 
fillo  versatility  planning  context.  Port- 

^ ot  optliMl  mixes  (portfolios) 

of  risky  aMets  (e.g.,  coMon  stocks)  by  rational,  risk-averse 
Investors  (Modigliani  and  Pogue,  1974).  Ue  will  treat  weapon 
systeM  as  risky  assets,  whose  performance  varies  according  to  the 
operational  situation.  Our  Portfolio  Theory  approach  leads  to 
mixed  pmurement  solutions,  and  provides  a conceptual  basis  for 
undertaking  Versatility  Planning. 

2 .„d  Illustrate  by  example  the  arguments  in  Sections 

.«!!!!“  u ^ consider  two  policy  implications  of  this 

pproach.  First,  we  will  use  the  portfolio  method  to  demonstrate 

acceptable  development  cost  of  a 
system  with  knw  performance  and  non-development  cost  character- 
Istlu.  Second,  we  will  consider  the  model's  Implications  for 
Intelligence  estimates.  Ue  believe  that  intelligence  data  should 
be  processes  to  emphasize  the  range  and  likelihood  of  various 

future  technical,  operational,  threat,  and  political  variables 

rather  than  Just  produce  most-likely  estiautes. 


2.  SOME  CURREST  DIFFICULTIES 

Procurement  decisions  are  never  made  In  a vacuum.  We  acquire 
weapons  systems  In  response  to  operational  needs.  Decisions  to 
acquire  and  operate  new  systems  oust,  therefore,  fit  into  the  • 
general  defense  planning  scheme.  It  also  follows  that  out  acquisi- 
tion strategy  mist  be  responsive  to  the  overall  needs  of  the  de- 
fense establishment  and  our  national  policy  objectives. 

are  now  becoming  aware  that  defense  planning  must  system- 
atically provide  for  the  wide  variety  of  future  scenarios  in  which 
our  forces  uy  ^ve  to  be  employed.  The  objective  Is  not  only  to 
attain  capable  forces,  but  also  to  attain  "versatility"  of  perfor- 
mance across  the  entire  range  of  future  military  environments.  The 
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nane  for  this  Is  Versatility  Planning!  ^ concept  for  future  defense 
planning.  ^ ^ , 

Despite  its  inportance,  the  defense  planning  conminity  has 
not  yet  undertaken  to  provide  versatility  planning  analysis.  ThU 
view  was  stated  by  Seth  Bonder  in  his  Keynote  Address  to  the  37th  > 
Military  Operations  Research  Symposium.  . . ^ 

'■  * * “ • 

• • .Senior  defense  managers  arc  (or  should  v n.  } : 
be)  interested  in  retaining  the  availability  of 
options  for  defense  forces.  . . .the  way  cost-- 

effectiveness  analysis  is  practiced  in  the  DOD  - ^ 

t } doesn't  provide  this.  I an  not  aware  of  any 
decision-theoretic  efforts  to  describe  formally 

the  concept  of  versatility  planning  or  to  pre-  r . 

scribe  how  it  might  be  implemented.  (Bonder.  1976) 

For  a number  of  reasons » the  DOD  finds  itself  dealing  with'  \ 
the  procurement,  or  proposed  procurement,  of  •^igh-Low”  mixes.  We 
find  it  interesting,  therefore,  that  the  term  is  not  defined  pre- 
cisely in  current  literature.  The  basic  meaning  is  fairly  clear: 
the  purchase  of  high-capability,  bigb-cosc  itei»  combined  with 
lower  capability,  lower-cost  items  to  accomplish  the  sane  basic 
mission.  We  still  find  the  lack  of  precision  disturbing,  and 
consider  it  to  be  not  merely  a matter  of  semantics.  In  the  interests 
of  more  systematic  analysis  of  the  subject,  we  will  offer  the  more 
general  tern  "mixed  procurement."  since  any  two  systems  need  not 
be  of  a High-Low  character. 

Since  we  are  buying  systems  to  accomplish  the  same  mission, 
we  will  assume  that  the  systems  thus  procured  are  perfect  substitutes 
for  each  other.  For  purposes  of  this  discussion,  we  therefore  offer 
the  following  definition:  nixed  procurement  is  the  acquisition  of  ' 
two  (or  more)  systems  that  are  perfect  substitutes  in  any  given 
operational  environment. 

The  foundation  of  most  of  economic  analysis  of  defense  problems 
lies  in  the  solution  of  a constrained  optimization  problem,  which 
can  take  either  of  two  logically  equivalent  forms.  The  first  is 
constrained  maximization,  which  is  stated  as 

maximize  V(X.Y) 

subject  to  C(X.Y)  - C . (1) 

where  V is  an  effectiveness  measure. 

X and  Y are  military  inputs. 

C is  a cost  function  which  relates  the  inputs  X and  Y to 
resources  expended  (usually  dollars). 

and  C is  the  budget  limit. 
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*“  Deslgn-to-Cost  methods. 
irSon'I  i!  ^ It  is  suggested  as  a possible  criterion  for  analysU 
? ® Igqn«»»ic  Analysis  Uandbonit  as  "most  effectiveness  for  a 

(JlJJh  "■  "“MW  <«’■ 

rtiici,  “• 


minimize  C(X,T) 
subject  to  V(X,Y)  - ? 
where  V is  a required  level  of  effectiveness. 


■r  * , uo 


(2) 


®®*i8“"to-Requirement6  methods. 
(Bonder,  1976)  It  is  suggested  as  a criterion  by  the  Defense 
^on^c  Analysis  Cowell  (DEAC)  as  "Least  Cost  for  a given  level 

vf  i»  proposed  as  the  "Gain 

^^****  Hitch  and  McKean.  (Hitch  and  McKean,  1960) . • 

efflellir  H^r^****  *p®  ^®«i“iiy  equivalent,  in  tnat  both  generate 
efficient  solutions.  Every  constrained  minimization  solution 
Mn^s  a coMtraiMd  mazlmizatlon  solution,  and  conversely.  For 
example,  if  ^e  minimum  cost  of  attaining  1500  units  of  effective- 
ness is  one  billion  dollars,  then  the  maximum  attainable  effective- 
ness with  one  billion  dollars  is  1500  units. 

With  our  terms  defined,  we  can  now  state  our  first  malor 
conclusion:  Constrained  optimization  under  certainty  will  not 
support  mixed  procurement.  To  support  the  conclusion,  it  is  first 
necessary  to  examine  "perfect  substitutes"  in  a bit  more  detail. 

For  ^rfect  substimtes,  we  can  write  the  effectiveness  measure  in 
tne  loiloving  form 

Vf-r^+r^y  (3, 

where  is  effectiveness  in  the  ith  scenario, 

X and  y are  the  amount  of  Systems  X and  Y acqulreo, 

*^X*  ^Y  unit  effectiveness  of  systems  X 

and  Y,  respectively. 

Actually,  we  are  asserting  only  that  Equation  (1)  is  a suit- 
able proxy  for  decision-making.  The  significance  of  the  subscript, 

1,  for  the  effectiveness  measure  will  be  discussed  in  later  sections. 
We  can  graph  effectiveness  as  a function  of  amount  X and  Y as 
shown  in  Figure  1.  (Bilas.  1967)  Any  given  level  of  effectiveness 
can  be  attained  by  any  combination  of  X and  Y lying  on  a straight 
line  with  slope  r 

am  — i • 
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Let*s  now  consider  a constrained  maxiaization  problea  of  the  fora 
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where  p„  and  p_  are  Che  (known)  costa  of  aysti 

* * - • . u \ ; 

X and  Y,  respectively* 

C is  the  budget  llnlcatlon*  ^ ^ 

and  other  teras  have  been  previously  defined. 


(4) 


Craphically*  we  can  show  the  budget  constraint  as  in  Figure  2* 
where  the  budget  constraint  is  portrayed  as  a straight  line  with 


slope 


If 


* we  have  Che  situation  given  in 


J 


in  Figure  3,  where  the  highest  attainable  effectlvenesss  occurs 


at  the  point  (x.y)  • .£_ 


Sinilavly.  if  r^  p^^ 


then  the 


x_ 

highest  attainable  effectiveness  occurs  at  the  point  i*»y)  ■ (0,  Py  ) 
If  the  special  case  r„  p_  occurs,  then  the  lines  of  equal. cost 


and  equal  effectiveness  are  parallel  and  the  highest  attainable 
effectiveness  can  be  achieved  anywhere  along  the  budget  constraint 
line.  In  this  instance,  the  declsioo-Baker  nay  choose  to  purchase 
only  Systeu  X,  only  Systea  Y,  or  any  conbination  of  the  two  within 
his  constraint.  However,  he  would  never  pay  a positive  cost  for 
the  privilege  of  purchasing  a second  system.  This  establishes  our 
conclusion. 

Given  that  currently  established  acans  of  anlysis  will  not 
lead  to  a nixed  procurement  solution,  and  given  chat  we  are,  in 
fact,  engaged  in  nixed  procurement,  we  are  left  with  the  uncomfort- 
able conclusion  that  the  nature  and  numbers  of  the  systems  in 
current  mixed  procurements  are  determined  mainly  by  informal  Judg- 
ment, or  by  Informally  imposed  constraints. 
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3.  THE  PORTFOLIO  THEORY  APPROACH 


We  have  now  examined  the  scam'trd  constrained  optimization 
approach  and  concluded  that  it  cannot  analyze  nixed  procurements. 

We  have  also  seen  that  current  methods  are  also  inade<iuate  for 
Versatility  Pianningm  In  this  section,  we  propose  to  analyze  mixed 
procurement  within  a framework  for  Versatility  Planning.  As  adver- 
tised, our  approach  wiU  borrow  heavily  from  an  established  body 
of  knowledge  called  Portfolio  Theor.''. 

TOe  essentials  of  Portfolio  Theory  were  developed  by  Harry  M. 
^rkowitz  during  the  1950*8  to  describe  the  selection  of  optimal 
investment  portfolios  by  rational  risk-averse  Investors.  Trans- 
ited ito  positive  applicatioas  by  JaJMs  Tobin  and  others,  Markowitz's 
Meas  became  modem  capital  market  theory,  revolutionizing  the 
fields  of  investments,  finance  and  the  economics  of  uncertainty. 

Martowltz  developed  his  approach  to  portfolio  selection  as  a 
.problem  of  utility  maximization  under  conditions  of  uncertainty. 
Emptying  probability  distributions  to  estimate  the  likelihood  of 
various  returns  accming  to  a given  security,  this  methodology 
computes  and  employs  the  central  tendency  or  "expected  value"  as 
• the  expected  return  from  that  security.  When  choosing  among  alter- 
native investments,  one  logically  selecu  the  one  with  the  largest 
net  present  value.  However,  actual  results  may  not  reflect  the 
Mpected  value."  The  extent  to  which  actual  return  is  likely  to 
diverge  from  the  mean  or  "expected  value"  of  the  probability  distri- 
bution ^ a function  of  the  spread  of  that  distribution,  a spread 
measured  statistically  by  its  variance  or  standard  deviation 
(which  is  the  square  root  of  the  variance).  The  tools  of  mean  and 
variance  are  employed  to  forecast  expected  rate  of  return  and  to 
measure  uncertainty. 

Portfolios  are  defined  as  coobinations  of  two  or  more  assets 
(securities).  Conceptually,  a portfolio  could  be  developed  by  an 
investor  to  either  increase  expected  returns  at  a fixed  level  of 
acceptable  risk  or  to  decrease  the  risk  associated  with  attaining 
a target  level  of  expected  returns.  A portfolio's  expected  return 
is  the  weighted  average  of  expected  returns  on  its  component 
securities,  using  the  proportions  invested  as  weights  n 

(Vi-1  *i^i^* 

Its  standard  deviation  depends  on  the  standard  deviations  of  return 
on  the  portfolio's  component  assets,  their  correlation  coefficients, 
and  the  proportions  invest;cd: 

o h n 


% • ih 


or 


(5) 


where  is  the  covariance  between  the  i‘**  and  assets. 
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Tills  covariance  factor,  when  scrutinized,  yields  important 
insights  into  the  ability  of  investors  to  reduce  risk  by  careful 
selection  of  assets  to  be  included  in  the  portfolio.  Basically, 
assets  whose  rates  of  return  under  the  various  plausible  scenarios 
states  or  nature  are  not  too  highly  positively  correlated  can 
be  co^ined  to  form  portfolios  which  provide  a smaller  variance  of 
expected  return— less  risk — than  any  one  of  the  assets  taken  alone. 

We  can,  in  fact,  reduce  risk  through  careful  portfolio  selection. 

In  standard  portfolio  theory  risk  is  partitioned  Into  systcaatic 
variability  of  return  and  unsystcakatic  variability  of  return. 
Syst»atic  variability  is  caused  by  factors  which  ace  associated 
with  the  narket,  the  scenarios  which  nay  pertain  (e.g..  Inflation); 
unsystaatic  risk  is  definitionally  peculiar  to  the  asset  end  not 
associated  with  narket  forces  (c.g.,  poor  aanageaent  of  the  fim). 
Systenatic  risk  is  the  focus  of  our  inquiry  and  is  the  risk  that 
’ can  be  reduced  through  conbining  assets  which  are  less  than 
• fectly  positively  correlated  into  a portfolio.  A sinplistic 
exanple  involving  two  perfectly  negatively  correlated  assets  yielding 
returns  between  30  percent  and  zero  will  illustrate  the  fundamental 
concept.  Given  an  equal  investnent  in  each  asset,  the  return  pro- 
files are  shown  in  Figure  4:  ^ 
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(Insert  Figure  4) 
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While  the  return  on  each  asset  ranges  between  30  percent  and  zero 
as  the  various  states  of  nature  are  assumed,  the  return  on  the 
portfolio  is  a constant  15  percent.  This  is  due  to  the  perfect 
negative  correlation  assumed  in  the  example.  Mote  that  risk,  as 
measured  by  variance  of  expected  returns.  Is  zero  in  the  illustrative 
portfolio  because  the  expected  portfolio  returns  under  any  of  the 
possible  states  of  nature  is  15  percent.  This  graphically  demon- 
strates the  power  of  the  covariance  concept  in  reducing  risk 
(though  finding  perfectly  negatively  correlated  assets  is  not  an 
easy  task!) 

If  alternative  portfolios  are  examined  for  risk  (Op),  and 
expected  return  (Ep),  and  the  dominance  principle  applied,  an 
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efficient  frontier  energes.  That  frontier  consists  of  dominant 
portfolios  for  any  given  set  of  risk  and  return  coordinates  as 
illustrated  in  Figure  3: 


o 

L' 


(Insert  Figure  3) 
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That  is»  all  points  lying  below  the  efficiency  frontier  are  dominated 
by  a point  on  that  curve  %#hich  offers  more  return  for  identical  risk 
of  less  risk  for  the  same  return*  Only  undominated  portfolios  lie 
on  the  efficient  frontier*  Hie  portfolio  selection  problem  emerges  ■ 
as  a straight-forward  optimizatioav  which  involves: 

(1)  One  or  more  decision  variables  (i.e.,  n assets);  | 

(2)  One  or  more  constraints;  and  * | 

(3)  An  objective  function  to  be  maximized  or  minimized.  » 

* * I 

Each  investor  is  Intent  upon  securing  that  portfolio  which 
both  lies  upon  this  efficiency  frontier  and  is  on  his  highest  in- 
difference curve  which  possesses  a point  of  tangency  with  that  / 
efficiency  frontier. 


(Insert  Figure  6) 
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Given  the  curves  in  Figure  6,  the  investor  selects  portfolio  A 
because  it  maximizes  his  satisfaction.  In  the  section  which  follows, 
these  elements  of  portfolio  theory  will  be  applied  to  selection  of 
weapon  systems  which  serve  similar  purposes. 

The  objective  of  Versatility  Planning  is  to  acquire  a force 
wliich  is  not  only  capable,  but  which  performs  at  least  reasonably 
well  throughout  the  rawge  of  probable  operational  environments. 

In  Section  2,  we  spoke  of  evaluating  effectiveness  in  some 
ith  scenario.  In  any  given  operational  situation,  the  unit  effectlve- 

'^y»  known  (or  can  be  determined)  with 

certainty.  . The  point  is,  however,  that  we  don't  know  under  what 
•tnte  (or  scenario)  we  will  employ  these  forces. 

The  criterion  for  evaluating  any  force  structure,  therefore. 

Mt  relate  to  both  "overall"  capability  and  versatility  of  per- 
for^ce  throughout  various  scenarios.  One  criterion  of  this  nature 
has  been  proposed  by  the  Portfolio  Theorists,  who  evaluate  mixes 
(or  portfolios)  of  risky  assets  using  a criterion  of  the  form 

• X,  i ' 

V V I • 

where  ^ the  expected,  or  average,  return  of  the  portfolio. 


Op  is  the  standard  devlatliM  of  the  portfolio  return,  and  X 

Cl* -’is  a measure  of  risk  aversion.  When  all  other  things  are 

- ^ual  a larger  L and  smaUer  o is  desirable  (to  risk-averse  ' 
oeclslon  makers)*  ^ 

'S,  f 

^^cted  return,  is  a measure  of  "overall"  portfolio  capability 
while  the  nortfol'lik  ^ a.  . 


while  the  portfolio  standard  deviation^  Op, 
of  the  portfolio's  versatility.  ^ 


is  an  inverse  measure 


Intend  to  treat  various  weapons  systems  as  risky  assets 
(like  comaon  stocks) , which  perform  well  in  some  circumstances  and 
less  weU^in  others.  Therefore^  we  will  treat  the  uni."  effective- 
ness measures,  and  , as  random  variables  which  can  vary  according 


to  the  operational  scenario.  We  can,  therefore,  consider  the  unit 
effectiveness  measures  to  be  of  the  form 


and 

(«)  , 


wliere  0 is  a collection  (vector)  of  random  variables  which 
describes  the  future  state  of  the  world.  Various  parts  of  this 
collection  would  include  factors  relating  to  operational,  technical, 
^litical,  and  other  attributes  of  any  possible  combat  scenario. 
(Bonder,  1976)  Since  some  scenarios  are  more  likely  than  others. 
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TH^  P®”^**!*  «ates  of  « are  described  by  some 

Joint  Py®^a‘»*-^ty  function  which  measures  those  likelihoods. 

• With  this  data  in  hand,  we  can  then  treat  weapons  system 
Just  as  we  treat  risky  financial  assets.  Knowing  r^^  (❖),  r (❖), 

'^y  r . we  can  refonnulate  the  ' ' 

constrained  optlnization  probtes  as  follows:  . 

^ '•  ’ -n: -riwv.M.sitli, 


U - Ep  - 


•f'-  . V I •JLi's’  f/o^- 

- ' < •*  loj 


■ u.  . c ,•  .,=,^ v.,.^ 


Op  -jfx^o^(rjj)  + o^r^)  + m pCr^.r^)  a(r^)  o(r^)* 


(a  rewritten  form  of  Equation  5)^ 
where 


,.7 


‘ - -i : 


2 2 • . ' . - 

. i r .0  (r^)  and  o (r^)  are  the  variances  of  r^^  and  r^,  respectively. 


P(rj,  r^)  is  the  correlation  between  r^.  and  r^  (a  statis- 


tical measure  of  how  r^^  and  r^  vary  together), 

and  all  other  terms  are  previously  defined.  ' 

Ue. statement  in  (8)  now  puts  the  problem  in  portfolio  form.  ' 

optimal  solutions  using  the  same  methodology  demon- 
strated in  Figures  4,  5,  and  6.' 

L .1.  function,  U - Ep  - Ao-  , allows  systematic  planning  for 

both  force  versatility  and  overall  capability,  provided  we  have  the 


4.  AN  ILLUSTRATION 


suppose  that  we  wish  to  procure  fighter  aircraft  for 
the  air-to-air  mission,  and  that  our  choices  are 


cosQy,  large  airframe,  with  all-weather 

capability,  and 


y— relatively  inexpensive,  small  airframe,  with  very 
weather  capabilitym  ^ ^ 

B affecM^I  ®“PP®®®  ooly  one  variable  in  the  collection 

““  ““*>*  *'  »'  ‘»ter..c-aver..e 
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Relative  to  alnost  any  threat  we  would  expect  the  effective- 


ness (r^)  of  Syst 


X to  increase,  and  the  effectiveness 


Systea  Y to  decrease  as  w decreases.  Since  we  are  also  Interested  in 
resource  constraints,  we  are  particularly  interested  in  the  ratios  of 
effectiveness  to  cost.  We  can,  therefore,  plot  (fjj/Pjj)  aai  (r^/p^) 

as  functions  of  w,  the  average  air-to-air  visibility,  as  in  Figure  7. 

The  effectiveness  per  dollar  of  system  T Is  quite  low  at 
sero  visibility  which  increases  with  V. 

On  the  other  hand,  Systea  X is  highly  effective 
pcr_dollar  at  zero  visibility  and  becomes  incxeasingly  less  effective 
as  w increases.  The  two  curves  Intersect  at  w . If  we  wish  to 


attain  a aaxlaua  level,  of  effectiveness  at  some  fixed  cost,  and  if 
we  know  the  value  of_w,  then  we  will  never  engage  in  procuri 

If  the  known  w is  less  than  w , then 


li 


h 


2 


and  X is  ■pre_jco8t**effect:ive9  and  should  be  procured  exclusivelya 
If  w > , then  System  Y is  the  more  cost-effective.  If 

V - • then  we  are  indifferent.  lo  no  case  are  we  better  off 

with  two  systems  in  the  inventory  instead  of  just  one.  This  point 
was  made  in  Section  2. 

However g the  real-world  problem  is  that  we  doo*t  know  what  the 
average  axi-tu-air  visibility  will  be  when  the  force  is  committed 
(probably  because  we  don*t  kn«iw  where  the  force  will  be  committed). 
There  is  intelligence  and  climatological  data  available  to  give 
some  notion  of  what  w will  be  in  the  next  war.  He  can  suamaeim 
this  information  in  terms  of  a probability  distribution  for  Wg  as 
shown  in  Figure  8. 

If  we  wish  to  have  a versatile  force  we  can  choose  X and  Y 
so  as  to 


maximize  U ■ 


• Xo, 


^ -F  __ 
subject  to  + p^Y  ■ C • 

As  in  the  section  3g  this  problc 


can  be  graphically 


analyzed  as  in  Figure  9.  Since  obviously 

negative  correlatedp  the  efficient  set  bows  strongly  inward. 
Point  A represents  complete  purchase  of  System  Yg  Point  B all  of 
System  X;  the  curved  segment  AC  is  the  efficient  set  and  Point  D 
represents  the  optimal  solution. 
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5.  SOME  POSSIBLE  APPLICATIONS 


} 


This  section  will  consider  two  policy  Isplications  of  rh» 

ese:::-!— 

ayste-^n  develop,  'acquire,  and  operate  the  second 

the.  arl-  ^ procurement  entails  significant  costs.  Among 

* ' f-  ' • f 

«e  have  assu^J  tMl*®**"®  production  linw; 

e have  assumed  this  away  for  clarity  of  presentation; 

(2)  development  cost  of  the'addltionar^stM; 

(3)  various  overhead  costs  connected  with 
operations,  including  training  and  logistical  s^p^ru 

«e  have  to  expend  source  resources  to  bring  any  addi- 
operation,  the  question  we  wish  to  Consider  is 
•^»hat  is  the  maxij^  we  are  willing  to  spend  for  the  option?* 

includi  srT"  ’’®  •Joeotion,  we  will  expand  our  basic  model  to 
include  development  costs.  Ve  will  consider  tradeoffs  between 
development  and  procurement.  •^«*eoits  between 

System  X has  already  been  developed,  then  the  constrained 
optlmlxation  problem  in  Section  2 takes  tte  form 

. ■ I 

maximize  U ■ Ep  - AOp  . ‘ j 

subject  to  + p^Y.+  D(T)  - C ; 

where" D(Y)  - (0  if  Y - 0 


jOifY-O  - 

Id(Y)  if  y > 0 , 


and  D(Y)  is  System  Y's  (assumed  known)  development  cost. 
The  decision  to  expend  known  development  costs  on  a system  of 


or 


PxX-C 

Y - 

Px^  + PyY  - c - D(Y) 


Y - 0,  l.e.,  we  do  not  have  the  privilege  of  purchasing 
System  Y,  r * 


--  J 
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The  first  constraint  leads  to  a solution  at  Point  A in 
Figure  10a,  where  we  spend  all  funds  on  System  X and  attain  U ■ U . 
The  second  £onstraint  is  shown  in  Figure  10a  as  the  curved  sesaen^ 
BC.  Since  0(Y)has  been  expanded  and  therefore,  there  are  fewer 
funds  available  for  procurement.  Hence,  the  amount  of  System  X we 
can  purchase  with  the  remaining  funds  moves  in  to  Point  B.  In  this 
situation,  the  optimal  procurement  is  represented  by  the  Point  D, 
attaining  U - U2  . Since  > U^,  the  proper  decision  is  to  divert 

resources  from  procurement  to  developnent.  ' , I 

There  are,  however,  limits  to  the  amount  that  we'd  be  willing 
to  spend.  This  is  il.ustrated  in  Figure  10. b.  In  this  case,  the 
optimal  solution  (at  O')  from  the  second  constraint,  B'C*,  is 
clearly  inferior  to  the  solution  at  A.  There  is  a critical  value 
of  D(Y)  illustrated  in  Figure  xO.c.  at  which  we  ar-e  equally  well 
off  with  or  without  developing  the  new  system.  He  can  call  this 
value  D(Y)  The  value  D(Y)  may  also  be  considered  the 

. worth  of  the  new  system  in  our  w^Vkis  portfolio. 

The  value  of  depends  upon  the  characteristics  of 

System  X,  the  characteristics  of  System  Y,  the  degree  of  correlation 
in  performance  between  tbe  systems,  and  the  declsion-«aker's  degree 
of  risk  aversion. 

■Mathematically,  we  can  state  this  as  follows: 

where  Efr^)  is  the  average  per  unit  effectiveness  of 

X and  Y respectively, 

o(r^)  is  the  standard  deviation  of  the  per  unit  effec- 
tiveness measures,  ' 

rY)  is  the  correlation  coefficient  of  r^  and 
and  X is  the  pieviously  defined  measure  of  risk  aversion. 

If  we  take  the  characteristics  of  System  X as  given,  we  would 
expect  to  be  willing  to  pay  for  developing  a system  that  is  highly 
capable  [large  E(rY)J,  or  highly  versatile  (small  o(rY)l.  However, 

we  would  also  be  interested  in  developing  a system  whose  performance 
is  negatively  correlated  with  our  existing  system.  In  Figure  U.a., 
we  show  the  case  where  a system  slightly  inferior  to  X,  Y,  whose 
performance  correlates  perfectly  with  System  X would  never  be 
developed.  Even  with  zero  development  costs,  we  would  always  choose 
to  procure  only  System  X.  There  is  no  reason  to  develop  System  Y. 

In  Figure  11. b.,  we  show  a decidedly  inferior  System  Y^,  whose 

performance  is  highly  negatively  correlated  with  System  X.  System 
Y2  is  worth  a significant  development  expenditure. 
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In  practice  both  developnent  costs  and  performance  character- 
istics are  not  known  until  development  is  over  (and  the  funds 
already  expended).  However,  it  is  still  possible  to  estimate  per- 
formance and  costs  at  various  points  In  the  project.  (It  is 
theoretically  possible  to  derive  a Joint  subjective  probability 
distribution  of  performance  and  costs).  A key  variable,  however, 
is  the  performance  correlation  with  existing  systems.  Ue  V-w#  of 
no  systematic  effort  to  include  this  factor  in  our  development 
project  evaluation.  ‘ . 

B.  A very  real  question  in  response  to  our  analysis  is 
^ere  do  you  get  the  numbers?"  Ue  intend  to  get  our  numbers 
(probability  distributions,  etc.)  in  the  same  general  way  that  the 
^rtfolio  theorist  intends  to  get  his.  Portfolio  Theory  carefully 
differentiates  between  security  analysis  and  portfolio  ar.alysis. 
Security  analysis  is  an  art  which  entails  predictions  about  the 
range  of  future  prospects  of  financial  assets.  Portfolio  analysis 
* ■ethod  of  solving  a portfolio  acquistion  problem  using 
data  generated  by  the  securities  analyst.  (Sharpe,  1970)  We 
propose  to  use  the  Intelligence  analyst  as  our  counterpart  to  the 
portfolio  theorist’s  security  analyst* 

This  model  therefore  strongly  implies  that  intelligence  infor- 
mation must  be  the  starting  point  for  force  planning.  More  impor- 
tantly, it  has  major  implications  for  the  form  and  substance 
of  those  intelligence  estimates.  To  mesh  properly  with  the  model, 
various  perceived  threats  Mist  be  aggregated  into  groupings  based 
upon  operational  similarity.  For  example,  all  scenarios  positing 
a benign  air  superiority  environment  might  be  grouped.  Groupings 
should  have  significant  implications  for  systems  performance 
requirements  and,  therefore,  for  hardware  requirements. 

Each  threat  category  or  grouping  must  then  be  assigned  a 
likelihood  of  occurrence,  generating  a probabUity  distribution 
for  the  overall  threat*  The  developaent  of  an  explicitly  stated 
probability  distribution  is  not  currently  a function  of  the 
Intelligence  Community,  but  it  is  essential  to  employment  of  this 
' Portfolio  Theory  model  of  flexibility  planning  and  its  logical 
extension,  mixed  procurement.  Initially,  providing  such  data  will 
likely  prove  painful  for  professional  Intelligence  Officers,  but 
the  benefits  derived  fro*  fully  integrating  Intelligence  Estimates 
with  procurement  plans  will  more  than  compensate  for  the  near-term 
discomfort— force  planning  can,  in  fact,  begin  with  an  explicit 
consideration  of  the  threat,  as  planning  philosophy  always  contended 
that  it  should. 
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PtPgWEMElCT  PlAMimS  AT  TOE 
US  im  AWWCtT  MATEIUEL  BEADINESS  COWWND 


tobert  6.  Sce4i  * - * • ' ^ ^ 

Oeputor  for  ProcurMent  and  Production  ' • i.  . . > T*- 

US  Ai^  VwMt  Materiel  Iteadlaess  CttMnd  . . ' V • . ^ ;^y 

. '■  *"•  *11  ' 1.  ^.•.'  -r;- ^ -r-  uU^t 

In  ttw  Interest  of  presenting  a conprehensive  reriew  of  the  procureaent  planning 
processes  at  the  US  Arnr  Araaaent  Materiel  Meadlness  Caamd.  of  necessity*  I Mist  bMln 
by  providing  basic  Inforatlon  about  the  Coamd  aisslon.  Mjor  fiactlons*  and  special 
■Issloiis.  . , . • r - .1 

\ ‘ . li  -Hr  , . 

s * * . 

Ihe  assl9ied  alsslon.  sl^ily  stated.  Is  as  represented  on  this  chart.  Hie 
weapons  and  munition  program  represent  the  greater  part  of  the  current  annual  progrui 
of  appnxlnately  3.1  billion  dollars.  . . 

(Churt  1}  r.  , 

In  addition  to  the  oomxHty  areas,  AARON  has  special  ulssloa  assIgnMnts  which 
significantly  li^act  our  procureaent  planning  effort. 

(Chart  2)  ' ' ' . 

As  previously  stated,  the  assigned  logistics  aanageaent  responsibility  of  the 
Comand  largely  cncoapasses  a wide  range  of  aateriel  and  equipaent  related  prinnrily  to 
thp  weapons  and  ammltlon  required  to  support  the  authorized  force  structure.  This 
prephi cal ly  Illustrates  the  total  weapons  systeas  and  other  hanhtare  Iteas  that 
ARRCOM  HMiages. 

(Chart  3)  • ’ . " " 

* Ulth  the  basic  Inforaatlon  about  ARRCOM  behind  us,  I will  quickly  cover  the  pol1« 
guidance  flow  that  Inpacts  our  procureaent  planning  effort,  reoo^ilzlng  that  this 
guidance  fl«  applies  to  aost  buying  coaaands  represented  at  this  conference  today. 

As  you  know,  the  policy  guidance  develops  gross  requlrenents.  'Ta  following  chart 
depicts  the  factors  which  wst  be  addressed  to  establish  an  actionable  requlreaent  for 
procureaent.  It  oust  be  recognized  that  as  with  every  buying  comnand.  ARRCOM  Is  not  In 
a position  to  develop  the  total  naterlel  requlrenents  for  assigned  alsslon  Iteas  even 
though  requirements  computation  Is  a basic  responsibility  of  the  National  Inventory 
Control  Point  (NICP)  at  ARRCOM.  The  final  recoaaendatlons  are  developed  by  a coarittee 
of  representatives  from  various  organizations  and  project  managers  offices,  and  «ny 
tough  decisions  are  necessary  In  the  shaping  of  the  final  recommendation  because  all  Items 
iust  be  accommodated  within  the  program  total  dollars  guidance  provided.  Once  the 
buy  recommendation  becomes  part  of  the  applicable  budget  year  estimate  and  the  Department 
of  the  Amy  Procurement  Annex,  and  apportionment  Is  effected,  the  program  Is  released 
to  DA  by  OSO  where  It  Is  released  to  ARRCOM  through  OARCOM. 

(Chart  4)  ' . 

That  Is  not  to  say  that  from  this  point  forward  everything  Is  going  to  fall  Into 
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a nice  and  easy  procurewnt  planning  effort.  The  best  laid  plans  are  subject  to  change, 
and  ARROOH  like  every  other  buying  coaand.does  not  escape  the  iapact  of  changing 
custoaer  reguireaents.  late  release  of  prograns.  prograa  funding  changes,  cutbacks, 
stretchouts,  etc..  Just  to  naw  a few. 

The  Ar^  KateHel  Prograa  (Alf»)  and  the  Five  Year  Defense  Prograa  (FYDP)  ar«  basic 

to  set  the  procureaent  planning  processes  in  »t1on. 
tt»e  of  the  firU  jobs  to  be  done  is  to  break  the  end  itea/round  of  asaunition  into  its 
lilt  elects,  co^ute  the  quantities  required,  and  determine  the  separate 

fir  '"tolved.  A typical  co^NMicnt  breakout  for  a weapons  syste^aMnition 

Itea  1$  depicted  OT  tte  next  two  charts  (Charts  $ and  6).  There  nay  be  as  aany  as  55 
*^tote  procurable  elenents,  for  a single  round  of  awnmition.  and  an  equal  or  greater 

niaher  of  procurable  elenents  for  a weapon  end  itea.  ^ » urx.ier 

(Chart  7)  , 

ARRCOH’s  p^renent  planning  process  aist  address  aultfple  factors  to  assure  cost 

of  Ksi^  nateriel.  A key  docunent  in  the  planning 
“efewe  Plan  (FYDPf  idiich  you  all  know  provides  current  l«ar 
El  requirenents  for  the  subsequent  four  years.  To  this,  of  course,  awst 

be  added  other  000  and  International  logistics  custoaer  actual  and  projected  requirenents. 

l!lf  available  and  the  breakout  action  Is  coiwileted. 

*******  requireMts  will  be  procured  by  other  000  conponents.  what 
iilfllf’  owaercially.  and  uhat  aateriel  will  be  procured  throu^  our 

arsenal  and  anvinltlon  plant  production  cs^ilex. 

**  OMercially  procured,  it  oust  be  determined  how  it  will  be 
•**'*'^***d,  negotiated  co^titively  or  sole  source  or  restricted  to  planned 
tose  produars  pursuant  to  10  U5C  Z304(a)(16)  - Purchases  in  the  Interest 
of  National  Defense  or  Industrial  Hsblllzatlon. 

**•*  <tetoi^nation  is  made  that  the  aateHel  will  be  produced  by  the  arsenal  or 

amiMm  productim  plant  coeplex  due  to  the  unique  situation  that  there  is  United 

toctor  of  industry  to  produce  a najor  part  of  ARRCOH's  nateriel 
re^irenmts  • and-this  is  especially  true  with  nunitions  itens  - a totally  different 
«**""*"9  factors  cote  into  play.  All  planning  effort,  in  this  instance,  is 

capability  of  the  three  Goverotent- 
*"*I1*L*  ^ twenty-six  Govemnent-Oieied  Contractor- 

Operator  (GOCO)  anmition  plants  producing  or  capable  of  producing  programed  weapons 
and  ounltlons  requirenents.  ^ r 

benef^^^2l’'^lJX^fh]^!l^™?^’*™*'^‘^”  ^is  point.  I feel,  will  be 

Ptoducer  of  weapons  and  weapons  components. 
i«  gw  tubes,  nortars.  reooilless  rifles,  breech  mechan- 

**  * coabined  industry  and  govemnent 

fl^nitiM  propellants  and  explosives  are  manufactured  in  the  Any  anwnition 
?fuT»*  a»a*l^le.  obtained  from  industry.  Anmunition  aetal  parts 

'/*“*•  cartridge}  are  obtained  from  private  industrial  facilities,  with  hard- 

lote  not  attractive  to  industry  bidding  being  manufactured  in  the 
Pi*"**-  propellants  and  explosives  and  metal  parts  are  funneled 
teto  the  Any  load,  assexbly.  and  pack  (LAP)  plants  for  assembly  into  complete  rounds  for 

^ <Hrection  to  maintain  this  produrtion  complex, 
t^icil^kiiu*  Mintain  an  industrial  moblization  base,  retate 

temical  skills,  etc.  in  the  event  of  a national  emergency,  is  the  "Arsenal  Statute" 
vhich  was  originally  enacted  in  1920  as  an  amendment  to  the  National  Defense  Act  of  1916, 


i / 

‘kv  s2sJTpjiL“s^jr.2iSir^;?- » •«- 

/WRCOH*ir»llke^v’’olLII®i!fl*‘  *'’^*  point  th*  because  of  this  production  co^>1ex. 

Sntractor  “T"***.  ***•*“  •«  oP*™**  more  liferTprii 

S!e^^t  5^1*  •"  ^''•9«"te<‘  betKeen  other  OOD  canoneots. 

tovemaent  arsenals  and  aaaunltlon  plants,  and  private  Industry.  ^^wnen 

ltp«^  ^ capability  to  prodM  «rta1n^ 

AR^M  **'^*J9®^^'*<*  capability  exceeds  the  requireaents;  and  in  these  cases 

2K?i?i?S2n^ubi^S'tr«^  *i*®f*H"*’"* 

subjected  to  an  analysis  of  noneoononric  factors  such  as  ■)bi1ization 

asrr * “F  “m-cfen^T^^^^  X™pa^^i1i‘1^■ 

S «Ji!5.?tt“5S2«*,S'SrrSdTS!^  ''•«  “• 

fr  ^ planning  process  relative  to  arsenal  and  plant  Nortloading 

capability  and  Io«<est  "out-of-pockef  cost  to  the  Govern-  ^ 
^ occ^ions  when  the  requirenent  to  maintain  a viable  production 
the  ^ire  for  an  economical  production  rate  and  production  Sill  te 
provide  ^tinuity  of  production.  On  the  other  hand,  it  nay  be  deter- 

»d*l«^thS*^fl!Hiir!l®  cate  laitil  the  quantity  is  coqileted 

*llowi)?Lj!^’**^  inactive.  This  will  nomlly  happen  when  there  is  no  foresee^le 

«>"S<‘tecation  is  the  integration  of  procurement  with 
Planning.  Previously.  I referenced  the  use  of  10  (ISC  2304(a)(16). 

Kbil  zilion  wSiw  ^"‘•cest  of  National  Defense  or  Industrial”  ” 

*'l  probability  the  largest  user  of  negotiating  authoHty 
statid  ?l!r  logistics  responsibility  for  weapons  and  mnitions.  ^\s 

SmdSr.^1^^  "®  extensive  capability  in  the  pHvate  sector  of  industry  to 

nunftionrmtSH^tl^li^™*^  mntenel.  for  the  obvious  reason  that  our  weapons  and^ 

Svl  requirerents  are  non-connercial  and  negate  industry's  willingness  to 

wke  the  high  dollar  investment  in  facilities  and  equipment.  It  therefore  behooves  this 

5 &£ST“sr 

..d  p4d2.™r.T.?m"ntS*lf5SS»^ 

our  DllnIIinJ‘^eff5rfr'^T?“i‘*^if  ‘®  *“!!**  ®!I  availability  and  how  it  may  iiipact 

I*  happened,  and  I am  sure  it  will  happen  again,  that  due  to 
technological  advances  in  the  state  of  the  art,  ARRCOM  is  faced  with  the  fact  that  thA 

p2?til  ?e2stI^t““This  government,  is  nonexistent  or 

par^idiiy  existent.  This  situation  would  of  course  necessitate  either  the  rrt.nhUtihnmnt 

0,  lYSdXVij:  xxxs-. 

in  the  instance  of  a partially  existent  production  line.  Or  it  maTbSlhat  incrSed^ 


U 
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tt.  M,  «3iSL'^'Sv“  .‘JS.'rif  S2  2?iS“  g.”!""  «.■'>  "'"-y  ‘f ' 
titiS'i;  SStl^X  ®'^'pJJci!i5‘SM&*s?j;“5J2n,“  'if 

cwu.^.  !>.*....  ..wt.«'’r'pTS»,%'4S  r.j.t«”rii:,^^ 

rccauiMd  to  the  IssuoKO  of^leUittow^o'adiilii^'ftiii"*!]*^!**  *"?  ***® 

«l«dl.9  to  o>p««t,  oootroet  M JS,  tl»  ^wVLSSlffSiiS! 


.01  «“d't£TS'dfSS2  SIToftlSoi  f,iLS’"1?  SL'ff  “ ““" 

Sri:a.i  2t.'s.‘2yir.2«:s''%*2Ls’2f : ^ 

i!:ji%-roPs-.£!c":S'£r  H 

I^nedles. 

Ing  technlcal/functlonal  or  rellahllitv  ^ irtthout  Croat- 

appropriatelv  ®/M  ratings  on  contracts  being 

SioHSr- 

.^':?i2!!;trort^2sLTOB"f"r"“^^^ 

s»» 

procurement  of  the  as'si^ied  mteHel.  planning  effort  to  assure  effective 
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UNIQUE  ASPECTS  OF  ARRCOM 
PROCUREMENT  PLANNING  AND  ACQUISITION 

1.  MANUFACTURING  ARSENALS 

2.  ARMY  AMMUNITION  PLANTS 
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— U<PI«SftC?lTAT10^t  OF  OV?  ri^CTLAR 
A-109  IN  A WEAPON  SYSTEM  DF/CLOPMEMT 
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1£DR  Alvin  W.  Musgrave  Jr- , SC,  USN 
Assistant  for  Business  Administration 
Mti  Ship  Missile  Defense  Project  (PMS-404) 
Sea  Systems  Command,  Washington,  D.C. 


FOREWORD 


The  ultimate  goal  of  research  is  to  produce  results  which 
ewntually  find  their  way  into  beneficial  practice.  In  this  vein, 
this  paper  discusses  the  "implementation  end"  of  a landnarlc  pro- 
rarement  research  endeavor,  that  of  the  Commission  on  Government 
Procurement  ~ and  the  Commission's  findings  and  recommendations 
embodied  in  the  Office  of  Management  and  Budget  Circular 
A-109,  "Acquisition  of  Major  Systems." 

^Ae  discussions  contained  in  this  paper  are  based  on  the 
ex^rience  of  the  author  as  Business/Financial  Manager,  Shipboard 
Intermediate  Range  Combat  System  (SIRCS)  Project,  Kaval  Sea 
Systems  Command.  The  paper  does  not  reflect  original  research 
per  se  but  rather  provides  a somewhat  brief  overview  of  how  the 
SIRCS  Project  has  applied  the  principles  of  0MB  Circular  A-109. 

Tna  concludes  with  a number  of  personal  observations  on  A— 

109  and  the  weapon  system  acquisition  process  which  it  has  set 
out  to  reform.  Although  many  of  the  observations  that  are  made 
are  the  outgrowth  of  numerous  discussions  and  debates  with  my 
colleagues  over  the  meaning  and  effect  of  A-109,  the  opinions 
expressed  herein  are  solely  my  own.* 

The  foreword  to  this  paper  would  not  be  complete  without 
acknowledgement  of  those  who  have  been  key  contributors  to  the 
p^ess  on  which  1 am  reporting.  At  the  top  of  the  list  is  the 
first  and  current  SIRCS  Project  Manager,  CDR  A.S.  Mobley,  USN. 

CDR  Mobley  is  a tireless  and  dedicated  Naval  Officer  who  has 
married  a belief  in^the  benefits  of  A-109  with  an  incessant 
drive  to  provide  the  surface  Navy  with  a needed  capability, 
other  key  members  of  the  SIRCS  Team  are  too  numerous  to  mention, 
but  they  represent  an  unusual  assemblage  of  talented  and  hard- 
%iorking  Naval  Officers,  headquarters  and  laboratorv  civil  servants, 
and  contractors.  To  all  of  these  men  I am  indebted  fcr  the  oppor- 
tunity to  report  on  the  fruits  of  their  efforts.  Acknowledgement 
would  not  be  complete  without  recognizing  those  who  provided  the 
architecture  and  substance  of  the  Report  of  the  Coinaission'  on 
Government  Procurement  and  0MB  Circular  A^05I“  Both  documents 
contain  sage  guidance  for  those  who  engage  in  setting  the  course 
of  weapon  system  developments. 
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t u of  Office  of  Management  and  Budget  Circular  A-109 

rUnll^^  culminated  a nearly  2-year  joint  Administration  and 
Congressional  effort  to  establish  policy  guidelines  applicable 
mk  engaged  in  developing  or  acquiring  major  systems. 

The  Circular  A-109  is  a landmark  effort  which  adds  several  new 
dimensions  to  the  business  of  defining,  funding  and- pursuing 
major  pr^rams  in  fulfillment  of  basic  agency  roles  and  mission>. 

Svst^a'^^gxnJ^?^  the  Navy  Shipboard  Intermediate  Range  Combat 
System  (SIR^) , recognized  as  the  first  full-scale  attempt  to 
uplerarac  the  A-109  policies,  provides  an  excellent  opportunity 
to  ev^uate  a real-world  application  of  the  new  acquisition 

T^-  following  case  study  reviews  the  background  and 
development  of  the  SIRCS  program 
the  SIRCS  program  structure  relative 
to  A-109  wd  finally  provides  some  general  conclusions  in  the 
form  of  lessons  learned"  to  date. 


OVERVIfiW  ON  A-109 


In  November  1969,  Public  Law  91-129  was  passed  creating  a 
commission  on  Government  Procurement  (COGP)  for  the  purposes  of 
studying  and  recommending  to  Congress  methods  to  promote  the 
viconomy,  efficiency,  and  effectiveness  of  procurement  by  the 
^ecutive  Branch  of  the  Federal  Government.  The  Commission  on 
Government  Procureinent  was  made  up  of  Congressmen,  Senators,  and 
representatives  iren  the  private  sector  and  the  Government.  Al- 
most 500  people  worked  on  the  Commission  staff  conducting  the 
study,  and  over  150,000  pages  of  feeder  reports  were  submitted  by 
the  working  groups  to  the  Commission.  The  acknowledged  reputation 
or  several  of  the  Commissioners  and  exhaustive  research  and 
analytical  effort  conducted  by  the  Commission  staff  combined  to 
cause  generation  and  issuance  of  a 5-volume  report  that  has  set 
proMss^^  profound  and  needed  change  to  the  government  procurement 

U.S.  Senate  Committee  on  Government  Operations  of  the 
94th  Congress,  created  a Sub-committee  on  Federal  Spending  Prac- 
tices, Efficiency,  and  Open  Government  to  provide  a Senate  focal 
point  for  the  review  of  Federal  spending  practices,  particularly 
Federal  procurement.  A major  part  of  this  review  was  directed 
toward  the  report  of  the  COGP,  in  particular  those  recommendations 
made  by  the  Commission  concerning  the  controversial  area  of  ac- 
quiring major  syscems.  The  report  includes  12  specific  recommen- 
dations, which  in  summary  attempt  to  establish  effective  control 
over  systems  acquisition  programs,  define  organizational  roles 
and  levels  of  responsibility,  and  assure  decision  makers  have 
access  to  the  information  necessary  to  make  key  decisions.  The 
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Federal  Spending  Practices,  Efficiency,  and  Open  Government 
Subc^ittee  is  continuing  to  investigate  the  recommendations  of 
the  Commission  and  to  develop  a legislative  program  to  impleiKnt 
the  needed  changes.  • 

In  1974,  the  Congress  established  the  Office  of  Federal 
Procurement  Policy  (OPPP)  in  the  Office  of  Management  and  Budget 
(OHB)  to  provide  overall  direction  of  procurement  policies, 
regulations,  procedures  for  executive  agencies.  In  its  first 
^or  ^licy  action,  the  OFPP  issued  OMB  Circular  A-109  prescribing 
hw  major  systems  will  be  acquired  by  all  Executive  Branch  agencies 
Tnis^new  policy  was  consistent  with  the  unanimous  recommendations 
of  the  COGP  concerning  the  establishment  of  needs  and  goals  for 
new  programs,  exploration  of  alternatives,  choosing  a preferred 
system,  implementing  a system,  and  org^ization,  manageoent  i»"d 
personnel.  . . 

• - . A ^ 

y 

Circular  A-109  does  not  offer  a "cookbook"  approach  to 
systea  acquisition.  The  circular  recognizes  that  each  major 
system  acquisition  program  is  unique  in  terms  of  time,  cost^ 
technology,  management  and  contracting  approach.  Despite  these 
differences,  the  basic  process  and  the  principal  activities  in 
the  process  are  common  to  all  programs.  It  is  these  basic  ele- 
ments to  which  A-109  addresses  itself. 

first  key  decision  in  the  acquisition  process  is  the 
^nd  approval  of  the  mission  need  statement.  The  approach 
of  A-109  is  to  have  the  mission  needs  and  goals  evaluated  and 
reconciled  - - - in  the  context  of  the  overall  agency  mission, 
resources  and  priorities  — — — at  the  top  management  level  and  at 
the  outset  of  the  acquisition.  A-109  outlines  an  integrated 
systeutic  approach  for  establishing  mission  goals  in  terms  of 
capability  desired,  time  and  operating  constraints,  value  or 
worth  of  meeting  the  need  and  relative  priority  of  the  need 
within  the  agency,  thus  bringing  together  the  budget  and  procure- 
ment aspects  of  the  process. 

Beginning  in  Fy-79,  each  agency ‘will  be  required  by  law  to 
submit  budget  information  in  accordance  with  assigned  agency 
missions,  and  to  relate  agency-  programs  to  these  missions.  This 
requirement  is  part  of  the  Congressional  Budget  Act  of  1974  and 
is  reiterated  in  A-109  as  the  basis  of  establishing  an  earlier 
and  more  meaningful  communication  with  Congress  in  the  budget 
process.  The  objective  is  to  have  Congressional  level  issues 
regarding  needs  resolved  early  in  the  major  system  acquisition 
process,  before  the  commitment  of  major  resources  and  selection 
of  solutions.  Congress  can  thus  withdraw  from  the  annual  process 
of  detailed  review  of  system  solutions  and  program  acquisition 
data  and  can  focus  on  broader  questions  of  agency  roles  and 

missions  a role  more  appropriate  to  Congress'  fundamental 

purpose. 
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Aiwther  major  emphasis  of  A-109  is  the  early  initiation  of 
competition  and  maintenance  of  competition  in  the  acquisition 
process.  Competition  in  the  sense  of  A-109  is  not  merely  the 
solicitation  and  review  of  competitive  proposals,  but  rather  the 
use  of  competition  during  the  development,  demonstration,  ^uld,  if 
economically  beneficial  to  do  so,  throughout  the  complete  acquisi- 
tion process.  This  entails  the  actual  parallel  development  of  • 
several  viable  total-system  alternatives.  The  higher  cost  of  I" 
■aintaxning  this  competition  amounts  to  insurfutce  against  the 
possibility  that  premature  selection  of  an  alternative  may  later 
prove  to  be  unsound  or  unaffordable.  . -■  * , 


ORGAN! tAIWONAL  CHANGES 
" * OHDBR  A-109  • 
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The  Improvements  in  the  system  acquisition  process  called 
for  by  A-109  will  not  come  about  without  concurrent  changes  and 
redefinition  of  the  roles  of  the  various  activities  and  organi- 
sations involved.  Table  1 presents  a very  simplified,  perhaps 
oversimplified  comparison  of  the  A-109  roles  vis-a-vis  the  "tra- 
ditional* roles.  , ^ 

‘ - >1-,  ^ 

- r-r  , rr  v a/, 


SABLE  Is 


CONGRESS 


OSD 


SERVICE 


LABORATORIES 


IMDOSTRy 


TRADITIONAL  RCIS 


DETAILED  REVIBf  OP  • 
INDIVIDUAL  BUDGET 
LINE  ITEMS 

APPROVAL/DISAPPROVAL 
OF  SPECIFIC  PROGRAMS 

HARDWARE  ADVOCATE 


TECHKOLOGY/BAROWARE 

ADVOCATE 


PROVIDER  OF  RESOURCES 
TO  BUILD  HARDWARE  IN 
RESPONSE  TO  GOV'T  SPEC 


A-109  ROLE 


REVIEW  BUDGET  REQUESTS 
cm  MISSION  NEED  BASIS 


APPRO*/AL  OF  MISSION  NEED 


IDENTIFY  AND  ARTICULATE 
MISSION  DEFICIENCIES 

MAINTAIN  TECHNOLOGY  BASE; 
PARTICIPATE  IN  EVALUATION 
OF  SYSTDI  ALTERNATIVES 

RESPOND  WITH  SYSTEM 
ALTERNATIVES  (SOrn  OF 
WHICH  ARE  EVENTUALLY 
DEVELOPED  AND  PRODUCED) 
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assigned  a role  to  revi^  basis.  Similarly.  oSD  is 

articulated  by  the  individual  basic  mission  needs  as 

analyse  tSe  sLSfic  having  to 

of  the  system  to  be  acquired!  ^ goals  and  ultimate  purpose  . 

^ -3- 

in  which  systems  are  defined  and  with  respect  to  the  manner 

support  of  the  aoency's  mission«  oi  technologies  in 

between  R4D  effo?S^nd  ensuring  a linkage 

respond  with  projjfed  fvsJL  i®  assed  to 

expanded  opportunit;  tS  eSgigf  L !Lnf  “i^h  a greatly 

tions  as  op^sed  to  beinr?lnstr2i^d 

specificati^s.  ThrJelpSnSes  hardware 

firms  represent  alternni^  avshe^  a!  ® ^ competing  industry 

(along  with  others  that  mav  have  Knfn  d ®°®®®P^®  to  be  evaluated 
Identify  those  that  satisfv  developed  internally)  to 

defined  roles  provide  the^LellL^P'^?!!®**  ®2ed.  The  newly 

th.  public  »' 

overview  on  SIRCS  •..tf#’ 

=rH:SKSrr£saJ“!~ 

Missile  ^fense  TASMD°"system*^and^th  °"79°^®9,  Advanced  Anti-Ship 

S”3  s -isi::  bsssxSP 

s?  i=s.jn?:ssssms£ESz. 
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f requirciKnts,  cooibinin,  both 

A K«i  ^ surface  warfare  mission  needs  was  issued  in  May  1975  for 

range  combat  system  capable  of  defendir.i 
P*^oj®®ted  anti-ship  missile  threat  as  well  as  enoaaina 
threat  and  shore  targets.  The  new  effort  was  callcd^th*»  ^ 
Shipboard  Intermediate  Range  Combat  System  (SIRCS) • 

The  period  from  iiiid-1974  to  the  present,  coverina  siRcrs 

conceptual  formulation  activity,  paral- 
if  substantial  change  occurred  i" 

iTOSisitiii^^®  policy  affecting  the  Lvelopraent  and 

SlRCs^riiri*  ii  - well  after  initial  SIRCS  planning  commenced  - - 
SIRCS  program  planning  and  execution  actions  were  patterned  from 


'»rt 


SIRCS  ACQDISITION  STRATEGY 


^ ^ 1^.'  1 
ir  1 


program  was  initl?ited  with  a 3-page  statement  of  an 
operational  requirement  {mission  need)  for  a system  thanijld 
detection  through  engagement  capability,  modu- 
larly  adaptable  to  various  ships  according  to  individual  size 
mission  and  capability  constraints.  The  operational  requirement 
•?’®  of  the  threat  and  the  desired^perfor- 

' J^eli^ility  and  cost  goals  for  the  system.  This  concise 
statement  of  requirements  was  tailor-made  for  the  purpose  of 

ShS"lnd^sr^  ^ industry  a very  broad,  but  bounded^P^oblem  to 

could  respond  with  independently  conceived  conceots 
® hardware  requirement  Jor  il^Jcate  In? 
®.P®®hulated  technical  or  conceptual  approach  ^ 
emphasize  that  new  developments  were  neither  required  or 
desired  unless  significant  benefits  would  result!  This 

statement  provided  industry  with  the  flexi- 
trade-offs  and  propose  solutions,  based  u^n 
analysis  of  the  problem.  The  Navy  challenge  to  industry 
substantive  industry  responses,  resulting  in  ^ 
three  separate,  funded,  cost-type  concept  formulation 

RCA  ^ The^9^Mnth"t-o^«-  McDonnell  Douglas,  Raytheon  and 

9-nonth  contracts  committed  each  contractor  to  develop 

his^Sl^sir^'^hr*’*'  ® integrated  system  based  up^n 

and  ^ requirements,  existing  developmental  effort 

and  available  or  emerging  technology. 

In  effectively  established  the  industry  competition  early 

in  the  conceptual  phase,  the  SIRCS  acquisition  strawy  is  desiJnL 
con^tition  as  long  as  it  is  beneficial.  The  Navy  will^ 
incremental  investments  in  the  most  prorais^g 
number  of  options  is  reduced.  Throughout^ the 
e elopment  process,  the  competition  serves  to  "keep  the  contractor* 
honest-  in  what  they  proposed  or  include  in  their  cLceptI!  Jhe 
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decision  to  use  existing,  aodified  or  new  developments  is  balanced 
between  each  contractor's  evaluation  of  the  advantages  (performance, 
cost,  reliability,  etc.)  and  the  competitive  incentive  toward 
economy  in  the  design  activities.  , w •; 

A very  important  aspect  of  defining  the  requirements  as 
implemented  in  the  SIRCS  program  is  the  "bounding  of  the  problem”. 
This  involved  providing  three  specific  types  of  additional  infor- 
mation to  the  contractors  in  order  that  they  could  better  understand 
the  problem  from  a Navy  perspective.  Pirst,  the  Navy  provided  an 
expansion  of  the  requirements  (Operational  Requirements  Expansion) 
in  terms  of  quantitative  goals  and  thresholds;  in  effect  providing 
a region  withM  which  the  contractors  could  perform  their  tradeoffs.  '' 
Second,  baseline  information  was  compiled  and  provided  so  that 
each  contractor  could  respond  to  the  requirements  under  a common 
defined  constraints;  such  as,  threat  and  target  parameters, 
operating  environment  (man-made  and  natural),  configuration  of 
the  future  fleet,  characteristics  of  existing  ships  and  mapon 
systems,  and  resources  available  for  Navy  test  and  evaluation. 

Third,  the  Navy  provided  a Cost  Analysis  Guide  to  articulate  as 
clearly  as  possible,  detailed  cost  elements  and  definitions, 
ground  rules  and  assumptions  for  cost  estimating,  and  formats  for 
presentation  of  cost  analysis  data.  The  total  of  the  data  furnished 
in  a 7-volume  Government  iomished  information  (GFI)  package  was 
supplemented  by  oral  briefings  on  technology  base  programs,  and 
the  establishment  of  a GPI  data  center  containing  over  2,000 
technical  documents,  study  reports  and  system  manuals,  all  of  which 
was  made  available  to  each  competing  contractor. 

Raving  thus  defined  and  bounded  the  problem  for  the  competi- 
tive concept  study  contractors,  the  Navy  solid ced  industry 
responses  to  a rather  simple  statement  of  %#ork  (SOW) . Essentially, 
the  SOW  reads  as  follows:  ' 

.* .undertake  an  indepth  study... to  develop 
a system  concept  to  meet  the..  Operational 
Requirement  for  (SIRCS)  and. . .develop  a 
formal  proposal  to  validate  this  system 
concept. 

Although  the  SOW  appears  on  the  surface  to  be  overly  general,  it 
is  consistent  with  the  generalized  statement  of  mission  need  in 
the  Operational  Requircmient.  Initially  there  was  some  feedback 
from  industry  indicating  a preference  for  more  specific  guidance. 
However,  once  the  study  contracts  commenced,  there  was  no  question 
that  contractors  were  prepared  to  take  a total  system  approach  to 
this  broadly  defined  total  problem.  The  Navy  very  specifically 
outlined  the  required  content  of  a 2,035  page  study  report  as  the 
primary  deliverable  under  the  contract.  The  contractors  were 
required  to  define  not  only  the  proposed  total  system  concept, 
but  also  to  identify  the  analyses  and  trade-offs  that  led  to  the 
concept  and  fully  justify  their  decision  processes.  The  initial 
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evaluation  of  the  concept  reports  indicates  that  all  contractors 
were  responsive  to  the  requirements  and  that  the  concept  formula- 
tion phase  was  successful. 


Three  other  hey  elements  of  the  SIRCS  solicitation  based 
upon  A-109  policies  are  recognized  as  having  contributed  to  the 
success  of  the  SIRCS  concept  formulation.  In  keeping  with  the  A- 
109  policy,  draft  copies  of  the  solicitation  documentation  was 
distributed  to  industry  for  comment.  Many  substantative  comments 
%#ere  received  and  incorporated.  Another  feature  of  the  solicita- 
tion was  the  identification  to  the  contractors  of  the  criteria 
that  TOuld  be  used  to  evaluate  the  proposed  approach  to  the  study 
as  well  as  the  criteria  for  evaluation  of  the  resulting  concepts. 
This  also  served  as  a method  of  conveying  to  industry  the  rela- 
tive importance  that  the  Navy  ascribed  to  performance,  cost 
and  risk  elements  of  the  contractor's  concepts.  Finally,  the 
Navy  required  as  a deliverable  at  the  conclusion  of  the  concept 
phase  (and  plans  to  require  at  the  conclusion  of  each  subsequent 
program  phase)  a proposal  for  the  next  phase.  The  contractor 
concepts  deemed  appropriate  for  further  development  can  thereby 
be  continued  through  a series  of  carefully  planned  incremental 
investments  by  the  Navy. 


SYSTEM  AFFORDABILITY 


Balancing  the  degree  of  technological  advancement  to  yield  a 
given  operational  capability  at  an  affordable  cost  represents  a 
significant  challenge  for  any  modem  weapon  system  development. 

The  problems  involved  in  such  balancing  are  particularly  acute  for 
a'’  system  as  multi-dimensional  and  operationally  complex  as  is  the 
case  for  SIRCS.  The  challenge  is  compounded  even  further  due  to 
the  physical  constraint  of  SIRCS  being  a point  defense  vice  an 
inter-ship  or  force  coordination  type  weapon  system.  For  a point 
defense  system  such  as  SIRCS  there  is  an  inherent  conflict  (trade- 
off) between  achieving  cost-effectiveness  for  a given  platform 
versus  that  for. a total  fleet.  For  instance,  a relatively  minor 
but  needed  increment  of  system  effectiveness  for  each  SIRCS  suite 
may  multiplicatively  drive  total  program  cost  to  a level  which 
is  unaffordable  in  quantity. 


Recognizing  the  general  tendencies  toward  high  cost  system 
alternatives,  plus  the  criticality  of  concept  formulation  trade- 
offs to  the  achievement  of  a low  cost  design,  the  SIRCS  Project 
Office  established  a rigorous  cost  discipline  at  the  start  of 
concept  formulation.  Beginning  with  the  Operational  Requirement 
(OR) , acquisition  cost  goals  were  established  on  the  basis  of  10% 
of  ship  replacement  cost  or  $10  million.  Although  such  a cost 
goal  structure  was  crude  it  provided  a convenient  sliding  scale 
establishing  upper  cost  limits  for  various  configurations  of 
SIRCS  to  meet  differing  ship  mission  requirements.  When  aggregated, 
ship  platform  cost  goals  for  various  SIRCS  configurations  provided 
concept  formulation  contractors  with  a cost  region  within  which 
alternative  concepts  could  be  explored. 
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life  ““  cecoepese  t.t.l 

i'^sSo^on"  ■ 

rc„ire4.e,  i„ 

a^uisltion  cost  goals.  Second,  an  Operational  Reauiremonrs 

"‘®  following  direction  regardiSJ  H??  Sclc 

ss‘£sts'r\“  “V"“-  ™e~ 

J!rL^n^  /"  mission  vice  equipnent  parameters! 

values  were  stated  as  goals  and  thresholds,  thus 
^oyiding  an  interval  within  which  trade-offs  to  minSlzriife  ^ 

cycle  cost  could  be  performed.  To  add  specificity  to  this  ooliov 

79^1ife®e^clf°fi'i-^?  Analysis  Guide  enumerated  and  defiie? 

79  life  cycle  cost  elements  to  be  used  as  a basis  for  all  Mn- 

system  cost  estimates.  In  addition  to  this 
standard  cost  element  structure,  costing  ground  rules  * 

m4  output  fo^it.  were  provided  in  thS’c?.?  ta.Sn'c“lr^  ' . 

The  most  important  reflection  of  the  Navy'p  resolve  to 

affordable  design  for  SIRCS  was  tS  language  of  the 

The^telectiM^SitLla^  validation  phase  contractors. 

coStSctI  MnSin!5^fho  5^??  concept  formulation 

following  statement:  "System  Militarv 

ISs^^dea^riSn*^^  importance...".  System  Cost 

was  described  in  the  evaluation  criteria  as: 

,'***  • J-  r» 

gystere  Cost  - The  purpose  of  this  element  is  - / *' 

to  evaluate  the  total  life  cycle  cost  to  the 

of  acejuiring  and  operating  the  system  *•:  ♦ 

proposed  by  the  contractor.  . • 

In  the  Validation  and  Demonstration  Phase,  design  to  cost 

for  both  life  cycte  cosJ^S  u^r 
production  cost.  A series  of  LCC  goals  vill  be  established  for 
suites,  e.g.  high  value,  major  combatant,  and  other. 
Production  cost  goals  will  be  established  for  significant  hardware 
si!?!™*®'  SIRCS  weapons,  surveillance  and  tracking  sub- 

control  subsystem,  etc.  This  structure  will 
provide  competing  prime  contractors  with  some  flexibility  to 
continue  to  optimize  their  design  with  respect  to  LCC  while 
giving  the  Navy  a measure  of  control  over  unit  production  cost. 

^ made  a part  of  the  SIRCS  specification  for 
^ validation  phase,  with  both  cost  and  performance  parameters 
specified  in  a manner  to  give  contractors  sufficient  latitude  for 
assets  of*^SI^S  simultaneously  validating  the  high  risk 
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and  inportant  change.  The  followina  nif k?^***™' **  **®  significant 

PRESSURES  TO  MAKE  PREMATURE  ’ ''  ' ' ‘ •**  ‘ ^<»i  3 5>r>2Q?c4*^. 

CHOICE  AMONG  ALTEr-TATTVPg ' ' trtxlnv' 

*■  ■ •'  ' r 'Ll  '«  ’>  C-t:  ?«>  *e»4-:'i7.T- 

Premature  weapon  system  choice  is  aii  area  ♦•Ka*-  SuJ 
on  Government  Procurement  singled-out  fj?  ?e£ori*^ 
report  succinctly  stated  the  ?oot  of  th^proSlJ;  *‘®  Commission's 

.TO  explore  different  system  concepts  and  V’^  ' 

,.,  introduce  competitive  development  requires  R&o  ,•» 

not^^made  av2?lIbL  M'  ' 

^ until  a decision  has  been  reached  that  a • - w- 

given  system  approach  should  be  pursued.  * ■'•  ' ‘ ‘ 

something  of  a paradox. ^ 

RAD  funds.  Beyond  this*  th2J*i-*in  *'ignificant 

.hlch  creates  ^ 

Sent^o'^schad)!?.  ^ management  discipline  InTln  il^li- 

^noeorc  f^fnificant  pressure  to  pursue  a rapid  winnowing  of 

SSL*  of tSfJSriLLa^S! 

TIME  AND  RESOURCES  SPENT  ^ ' 

IN  FRONT-END  SOURCE  SELECTIONS 

a broad  base  of  alternate  conceots  and 
during  the  Exploration  of  Alternatives  PhJse  (cL- 

short-term  low  dollar  value  con- 
iSain  f to  provide  flexibility  for  pursuing  an  uncon- 

strained and  Iterative  search  for  the  most  promising  alternatives 

^ *^oo  commission  on  Government  Procurement,  Vol.  2, 

Cp  p«  89 • 
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U>  aeet  a mission  need.  Such  contracting  flexibility  would  mean, 
for  example,  that  contractot s/concepts  would  be  dropped  and 

based  on  periodic  new  information  stemming  from 
* concept  formulation  and  exploratory  contracts.  Such 

an  approach  makes  a great  deal  of  sense,  but  does  not  appear  to 

resources  which  would  be  claimed  under  the 
te^tional  Department  of  Defense  (DOD)  approach  to  source  selec- 


One  could  argue  that  A-109  series  contracting,  as  described 
a^ve,  does  not  entail  source  selection  per  se.  Prior  to  A-109 
the  concept  formulation  phase  frequently  entailed  low  dollar 
value  contracts  with  potential  developers.  A source  selection 
'^®®  avoided  because  a validation  phase  contractor  or 
TOnteactors  %rere  noisily  selected  as  a result  of  a new  solid- 
ution,  with  the  basis  of  the  solicitation  being  developed  during 
mncept  formulation  phase  through  the  melding  of  various 
inputs  from  both  government  and  industry  sources.  The  approach 
however,  is  to  avoid  such  melding  and  resolicitation, 
and  to  instead,  substitute  parallel  comp>etition.  Hence,  instead 
of  periodic  source  selections,  A-109  tends  to  create  a continuous 
Murce  selection  process,  particularly  during  the  front-end 
E^loration  of  Alternatives  Phase.  As  a consequence,  each  deci- 
sion to  continue  a competitor,  and  more  importantly,  to  drop  a 
contractor  becomes  a source  selection  decision.  For  major 

such  decisions  are  of  course  of  major  importance  to  each 
";®'>®‘^rial  participants.  Thus,  a dilemma  is  created  by 
tte  A-109  parallel  competition  approach.  For  major  programs, 
where  exploration  of  alternative  concepts  should  normally  be 
pursued  to  the  maximum  extent,  there  may  be  a counter  influence 
to  minimize  such  exploration  because  of  the  time,  expense, 
and  trouble  associated  with  the  source  selection  machinery  which 
DOD  policy  VK}uld  dictate  must  be  established. 

The  answer  to  the  above  problem  relates  to  the  manner  in 
which  the  current  DOD  source  selection  policies  are  carried  out. 

beyond  ^he  scope  of  fhis  paper  to  engage  in  a discussion  of 
^^^stinq  DOD  source  selection  practices.  However,  the  problem 
discussed  above  warrants  scrutiny  of  source  selection  conventions 
with  a view  toward  creation  of  a more  streamlined,  less  formal, 
and  less  time-consuming  process,  and  one  which  can  truly  accommo- 
date the  unfettered  exploration  of  concepts  from  contractual 
sources  as  called  for  by  A-109. 

HIGH  COST  OF  EXPLORING 

ALTERNATIVE  CONCCPTS 

Developing  suitable  information  for  a spectrum  of  alterna- 
tives to  fulfill  a major  system  deficiency  is  expensive.  With- 
out adequate  budgetary  support  on  the  "front-end"  the  base  of 
alternative  concepts  will  probably  have  to  be  artifically  limited. 
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This  was  the  case  in  the  SIRCS  Prograsi,  with  a lack  of  budge- 
tary support  forcing  a limit  in  the  number  of  concept  study 
contracts  %rhich  could  be  awarded/  The  problem,  ho%#ever,  is  even 
more  basic  than  simply  a lack  of  funding  for  study  contracts. 

I r.  • { 

Under  A-109,  a vastly  expanded  scope  of  activites  is  to  be 
conducted  in  defining  requirements  and  exploring  alternatives. 

This  new  scope  of  effort  in  initiating  programs  will  require 
additional  resources  not  only  on  the  government  side,  h* *it  there 
are  also  implications  for  the  manner  and  level  at  whic«.  corporate 
independent  research  and  development  (IRfcD)  f vends  are  expended. 

One  can  speculate  that  as  000  becomes  more  a:customed  to  the  A- 
109  process  that  R0T4E  rer^ources  will  become  hore  properly  allocated 
as  a result  of  DSARC  Zero  decisions.  Similarly,  one  could  expect 
that  corporate  IRiiO  monies  and  other  internal  fundj  will  become 
more  appropriately  channeled  in  response  to  documented  statements 
of  mission  need  em£LnatiD9  from  the  Services.  Notwithstanding 
these  predictions,  the  point  is  that  A-109  requires  an  expanded 
effort  for  early  program  activities.  Funding  tor  cnis  new  require- 
ment dictates  an  appropriate  awareness  on  the  part  of  government 
and  industry  officials  in  the  conduct  of  R4D  resource  allocation 
decisions.*  , , . ^ 

ESTABLISHING  COST  BOUNDS 

PRIOR  TO  HARDWARE  DSFINjTION  / 

• A^109  requires  that  program  (not  unit)  cost  objectives  be 

contained  in  the  initial  mission  need  solicitation.  For  the 
SIRCS  Program,  a General  Accounting  Office  study  criticized  the 
approach  of  assigning  initial  design  to  cost  objectives  at  the 
platform  level  vice  the  tot»i  orogran  level.  Although  A-109  is 
theoretically  correct  in  the  approach  of  using  total  program  cost 
objectives,  practical  problems  make  full  implementation  difficult 
if  not  meaningless.  For  instance,  for  most  major  system  programs, 
with  the  exception  of  space  emd  satellite  systems,  the  number  of 
units  produced  is  the  most  significant  determinate  of  total  pro* 
gram  cost.  Additionally,  difficulties  arise  in  describing  the 
mission  capability  that  is  required  in  terms  other  than  those 
which,  de  facto,  tend  to  describe  either  the  unit,  the  platform 
from  which  the  unit  is  to  be  operated,  or  the  geographical  limits 
of  the  unit*s  operation.  For  SIRCS,  the  mission  need  that  SIRCS 
was  responding  to  was  described  in  terms  of  a system  operating 
within  a ship  to  provide  the  ship  with  a needed  capability. 
Trade-offs  and  system  alternatives  were  thus  constrained  to  the 
boundaries  of  the  single  ship.  It  viould  have  been  illogical  for 
the  SIRCS  Program  to  have  set  a cost  objective  for  a total  pro- 
gram without  one-to-one  correspondence  between  such  an  objective 
and  the  mission  need  to  be  fulfilled. 

Notwithstanding  the  express  direction  in  A-109,  the  most 
meaningful  way  to  address  the  problem  of  setting  initial  cost 
objectives  is  to  juxtapose  the  cost  objectives  with  the  unit  for 
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deficiency  is  described.  The  cost  objective 
TOUW  then  derive  from  a total  program  affordability  projection 

assumed  multiple  relating  to  the  unit  for 
mission,  deficiency  is  described.  For  example,  afford- 
projections  have  been  made  in  the  SIRCS  Program,  in 
asseping  design  to  cost  goals,  this  total 
be^outf  divided  by  a presumed  number  of  ships  to  ' 

giving  an  upper  cost  boundary  for  a SIRCS  suite. 

.-.w,  . T * *•  • - s 


IS  ^e  type  of  approach  described  above  for  setting 

affordability  projections.  Although 
laLSl  projections  have  represented  extrapo- 

b^getary  data,  affordability  should  be  based  on  a 
Si  *'‘’®  required  capability.  In  practice. 

SSSSli  operate  on  Lch  a higter  • 

mlesfon!iif*  implementation  of  A-109  and 

^ budgeting,  as  is  required  for  FTf  1979,  allocation 

ofresourros  based  on  a willingness  to  pay  for  a capability  may 
a routine  reality.  Indeed,  such  a reorientation 
in  ctellenge  in  the  i]iq>lementation  of  A-109  and 

in  bringing  about  an  integration  between  the  DSARC  and  PPBS 
processes 


INITIAL  COHCLWIONS  ON  A-i09 


c 


h WBAPOW  SYSTEM  DEVELOPMENT  APPROagw 


, ,,,  Experience  in  the  SIRCS  Program  has  given  rise  to  several 

o?a-109^  tentative  conclusions  regarding  the  long  term  benefits 


••  1.  ^Inqproved  linkage  of  resource  inputs  and  program  needs 
. 2.  Improved  atmosphere  for  innovation 


3. 

4. 


Reduction  of  suboptimization  potential 
Improved  system  engineering  discipline 
Reduced  development  lead  time 


*-h«  cTSlc"!**  definition  of  requirements  in  terms  of  mission  need, 
^e  SIRCS  Program  has  insured  that  development  dollars  expended 
will  te  linked  to  a valid  fleet  requirement.  Similarly,  the  Pro- 
gram has  effectively  insulated  itself  from  the  pressurL  of  the 
solution  looking  for  a problem",  syndrome.  As  a result  of  the 
January  1977  revision  to  DODD  5000.1,  an  important  step  has  been 
taken  to  institutionalize  such  advancement  in  the  resource  allo- 
cation process.  This  revision,  which  included  a provision  for  a 
Mission  Element  Need  Statement  (MENS)  as  the  basis  for  program 

serve  as  a key  instrument  forimproved 
linkage  between  resource  inputs  and  service  program  needs. 
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The  SIRCS  Program  has  enjoyed  an  atmosphere  of  inventiveness 
and  problem  solving  fostered  by  industrial  competition  and  the 
promise  of  significant  future  business.  From  this  atmosphere 
have  emerged  weapon  system  concepts  reflecting  an  objective  balance 
between  new  and  existing  technologies^  and  across  cost^  performance^ 
and  risk  diinensions.  Certain  features  of  these  system  concepts 
are  innovative.  Other  acquisition  approaches  may  have  achieved 
similar  results.  However « unquestionably  programs  following  A-109 
create  an  atmosphere  which  encourages  a high  incidence  of  aggressive 
and  innovative  problem  solving. 


Defining  requirements  in  terms  of  a total  need  forces  total 
system  solutions.  Trade-off  opportunities  that  might  not  be  ob- 
vious for  a subsystem  development  suddenly  become  manifest  and 
imperative  when  solving  the  total  system  problem.  The  potential 
pay-off  of  defining  total  systems  is  more  cost-effective  military 
systems  through  avoidance  of  inbalances  in  subsystem  designs^ 


( v: 


Perhaps  the  biggest  pay-off  which  SIRCS  has  enjoyed  under 
A— 109  is  the  disciplined  system  engineering  process  that  industry 
program  management  teams  have  applied  to  a complex  systems  problem. 
These  industry  teams  have  had  at  their  disposal  data,  information 
channels,  and  the  necessary  tools  to  engage  in  a timely  and  un- 
fettered rational  trade-off  and  sequential  decision  process.  The 
in-house  Navy  bureaucracy,  with  its  highly  differentiated  head- 
quarters and  laboratory  organizations,  would  severely  hamper  such 
a streamlined  and  rational  system  engineering  process.  Hence,  by 
going  to  industry  for  concept  formulation,  a system  engineering 
resource  is  acquired  that  would  be  very  difficult  if  not  impossible 
to  engender  within  the  Navy. 


Several  features  of  A-109  combine  to  offer  promise  of  a 
reduction  in  development  lead  time.  Under  A-109,  proposals  for 
the  next  program  phase  are  received  as  contract  deliverables. 

This  forces  advanced  planning  and  eliminates  the  need  for  genera- 
tion of  new  solicitations.  Parallel  competition  provides  schedule 
insurance  in  the  form  of  fall-back  options.  Most  importantly,  a 
well-reasoned  "front-end"  process  with  program  initiation  predi- 
cated on  secretarial  approval  of  the  basic  program  need  should 
give  programs  a sounder  charter,  more  adequately  programmed 
funding,  and  overall  an  improved  opportunity  to  pursue  an  effi- 
cient development  schedule. 


The  SIRCS  Program,  the  Navy's  experiment  with  the  A-109  ap- 
proach to  systems  acquisition,  has  progressed  through  an  initial 
stage.  Undoubtedly,  with  further  experience  in  SIRCS  and  other 
programs  following  A-109,  lessons  will  be  learned  for  the  best 
structuring  of  programs  following  this  new  acquisition  policy. 

Such  lessons  learned  are  of  crucial  importance  because  in  the  im- 
plementation phase  policy  reform  often  falters.  One  can  justifi- 
ably criticize  certain  aspects  of  A-109,  but  such  criticism  relates 
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to  inplemcnta^on  provisions.  Resolving  these  difficulties  is  the 
challenge  facing  the  DOD.  Failure  to  meet  this  challenge  means 
impeding  a policy  reform  that  is  long  overdue  and  that  offers  great 
promise  for  a system  in  dire  need  of  responses  to  its  fundamental 
problems. 


POSTSCRIPT 


The  Congressional  Record  of  June  20 p 1977 p containing  the 
Conference  Report  on  IT.R.  5970p  Department  of  Defense  Appropriation 
Authorization  Actp  1978  revealed  the  following  recommended  restric- 
tive language  for  the  SIRCS  Program. 


Of  the  funds  authorized  to  be  appropriated  under 
section  201  for  the  Navy  (including  the  Marine  Corps) 
for  researchp  development,  testp  and  evaluation,  an 
amount  not  to  exceed  $3 , 894 p 000  shall  be  available 
^or  (1)  defining  a set  of  design  specifications 
for  the  Shipboard  Intermediate  Range  Combat  System 
(SIRCS)  program,  and  (2)  conducting  an  open  compe- 
titira,  to  be  conducted  after  such  design  specifi- 
cations have  been  defined  and  to  be  based  on  such 
specifications p to  select  a contractor  or  contractors 
for  the  advanced  development  phase  of  such  program. 

In  developing  such  design  specifications,  the  Secre- 
tary of  the  Navy  shall  include  the  best  features  of 
the  designs  developed  by  the  three  contractors  which 
have  been  selected  for  the  program  before  the  date  of 
enactment  of  this  Act  and  such  other  features  as  the 
Secretary  considers  appropriate*  A contract  entered 
into  under  the  competition  required  by  this  section 
may  be  for  development  of  the  entire  system  or  for 
development  of  any  independent  subsection  (sic]  of  the 
system. 


The  accompanying  rationale  for  the  Conferees*  action,  con- 
tained in  the  Conference  Report,  stated  the  following. 


The  SIRCS  program  has  followed  the  aeguisition 
strategy  of  the  Office  of  Management  and  Budget  Cir- 
cular A-109  and  intends  that  two  contractors  be  funded 
to  build  competitive  prototypes  of  a new  radar  and  new 
missile  integrated  with  a fire-control  computer  system 
for  shipboard  use.  The  Navy  presently  is  evaluating 
proposals  submitted  by  three  separate  contractors  for 
these  coi!^K>nents  • Under  the  program  as  presently 
structured  the  Navy  could  not  select  the  most  desirable 
cosqponents  from  individual  contractors  but  would  have 
to  fund  continued  development  of  the  systems 

of  two  of  the  three  contractors.  This  acquisition 
approach  could  cause  continued  development  of  less 
than  optimum  subsystems,  a factor  which  exposes  a 
potential  shortcoming  of  the  development  procedures  of 
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0MB  Circular  A-109.  Consequently^  the  House  conferees 
emphasize  that  the  language  specified  in  Sec.  203  is  a 
clear  intent  to  exclude  SIRCS  from  the  A-109  process 
and  made  the  funding  authorized  contingent  upon  this 
' condition. 

The  conferees  agreed  that  the  next  step  in  the  SIRCS 
program  is  for  the  Navy  to  define  a "baseline*  design 
containing  the  best  features  contained  in«  but  not 
-limited  to,  three  funded  contractors,  and  that  this 
baseline  SIRCS  is  to  be  submitted  for  opcm  competition 
for  the  advanced  development  phase.  Under  this  pro- 
cedure the  Navy  has  the  flexibility  to  select  or 
develop  independent  subsystems  such  as  the  radar  or 
missile  for  independent  contracts  if  such  action  is 
determined  to  be  desirable  and  will  provide  the  best 
possible  weapons  system. 

The  conferees  agreed  to  provide  $3.9  million  for  the 
SIRCS  program,  with  the  understanding  that  the  Navy 
uses  the  baseline  approach  to  continue  the  program. 

The  above  Congressional  direction  is  interesting  in  that  the 
criticism  of  the  SIRCS  Program  relates  only  to  the  qualified 
charge  that  "This  acquisition  approach  could  cause  continued 
development  of  less  than  optimal  subsystems".  Indeed,  a fundamen- 
tal objective  of  the  SIRCS  Program  is  to  develop  an  optimized  sys- 
tem not  optimized  subsystems. 

The  following  statement  in  the  Conference  Report  indicates 
that  the  Conferees  have  apparently  misinterpreted  the  flexibility 
which  the  Navy  has  in  managing  the  SIRCS  Program. 

Under  the  program  as  presently  structured  the  Navy 
could  not  select  the  most  desirable  components  from 
individual  contractors  but  %rould  have  to  fund  con- 
tinued development  of  the  complete  systems  of  two 
of  thh  three  contractors. 

This  statement  is  literally  true,  but  the  constraint  is  only  self- 
imposed  and  not  binding  in  so  far  as  management  flexibility  is 
concerned.  The  point  is  that  based  on  Navy  review  of  TOntrdetor 
concepts  there  is  no  current  justification  for  exercising  a peroga- 
tive  to  modify  the  basic  acquisition  approach  and  to  transfuse 
concept  features.  Should  further  development  activities  indicate 
that  certain  transfusion  is  in  the  best  interest  of  the  Navy, 
then  one  would  expect  that  appropriate  action  would  be  taken  to 
modify  the  acquisition  strategy.  In  reality,  there  is  a consider- 
able amount  of  beneficial  transfusion  of  best  features  as  a conse- 
quence of  the  forces  of  competition.  These  competitive  forces 
cause,  for  example,  competitors  to  select  features  deriving  from 
common  government  laboratory  and  corporate  sources,  and  for 
winners  and  losers  to  gravitate  to  one  another  if  there  are 
residual  best  features  in  a loser's  concept  which  would  enhance 
that  of  a winner. 


. In  response  to  the  restrictive  language  of  the  FY  78  Defense 
Authorization  Bill,  the  Secretary  of  the  Navy  has  appealed  to 
key  Congressional  leaders  stating  his  objection  to  the  action 
taken.  It  remains  to  be  seen  what  the  ultimate  outcome  will  be 
for  SIRCS  and  the  current  experiment  with  the  A-109  approach  as 
a consequence  of  final  Congressional  action  on  the  FY  78  Budget. 
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essentially  the  aaterial  found  In  Dr.  Hood's  award-winning  article  In 
^ HCHA  Journal.  Dr.  Hood's  talk  was  extc^wraneous  but  based  on  this  text. 


INTRODUCTION 


' Occupations  differ  frow  one  another  In  the  anount  and  kinds  of  further  education 

^ i^ulre  of  practitioners.'  (1)  Horeover.  the  diversity  of  orgwilzatlonal  situations 
- In  which  practitioners  of  specific  occupations  find  theselves  differ  fioa  one  another 
( ) In  avunt  and  kinds  of  preparation  required  periodically  for  effective  role  perfbrMce.(2) 

One  such  field  of  practice  with  such  dynawlc  diversity  Is  that  of  federal  procurewit— 
the  a^lvltles  perforwed  by  nany  people  In  different  organizations  which  acquire  systeas. 
supplies  and  services  for  the  agencies  of  the  federal  govemBwt.O) 


The  total  annual  federal  procurcaewt  expenditure  exceeds  60  billion  dollars. (4) 

. - The  size  of  this  expenditure  profits  people  to  be  concerned  about  the  Integrity  of  the 
/ , processes  through  which  the  expenditure  Is  aade.(5)  The  critical  co^xwwnt  In  the 

V^-  process  ^cem  the  knowledge  and  skill  possessed  by  personnel  working  In  the  procure- 
• went  offices. (6) 

/ I The  largest  agency.  In  terv  of  both  nudbers  of  people  working  In  procurenenC  and 

\ ■ I dollars  of  expenditure.  Is  the  Departaent  of  Defense. (7)  This  agency  established  a 
career  prograa  for  both  military  and  civilian  procurement  personnel  in  specific  terms 
^aalntalnlng  a conscious  connection  of  these  personnel  with  the  Institution  of  the 
^artment  of  0efense.(8,9)  The  Defense  Management  Education  and  Training  Board  estab- 
llshed  a procurement  training  program  specifically  for  civilian  personnel. (10)  Part 
of  this  prograa  provided  for  specific  job  oriented  courses  to  be  taken  during  the 
procurement  practitioners'  progress  from  Grade  GS-5  to  Grade  GS-13.(11)  These  courses 
\ “ "mandatory'' because  of  the  need  within  the  DOD  of  a training  program 

mIct  sufficiently  assures  a minimum  level  of  conpetenoe  of  Its  procurement  practitioners. 
(12)  Part  of  this  need  arises  from  the  diversity  of  educational  backgrounds  and  job 
, experiences  of  these  peop1e.(13) 

^ . Reports  are  periodically  made  showing  the  nueher  of  personnel  within  the  specific 

procurement  occupations  that  have  completed  the  mandatory  courses  required  for  their 
grade  or  rank.  A recent  report  reveals  that  less  than  50  percent  of  the  civilian 
procurement  personnel  have  taken  the  mandatory  courses *0^) 

This  surfaces  the  question  of  hhw  people  within  the  respective  ranks  acquire 
the  i^uisite  knowledge  and  skills  to  perform  their  assigned  duties  and  responsibilities 
\ In  the  procurement  offices.  Many  other  questions  are  subsequently  raised.  Did  th^e 
people  engage  in  some  sort  of  Individual  or  group  learning  effort  independent  of  the 
. esUblished  program?  If  they  had,  are  they  still  doing  so?  How  much  effort  do  they 
spend  in  such  learning?  What  is  the  content  of  their  learning?  What  assistance  did 
( What  difficulties  did  they  encounter?  Can  there  be  organizational  changes 

which  will  foster  this  kind  of  learning  effort?  Can  there  be  ar^y  changes  In  their 
preparation  for  learning  about  procurement  work  in  either  their  education  or  experience? 
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As  suggested  by  the  preceding  sections,  nine  researeh  questions  are  explored:  ^ 

ROi:  How  extensive  are  self-learning  projects  practice  aa>ng  precurenent  personnel?  - • 


Slgrlflcance  of  tte  Research 

the  anunt  of  tloe  they  spend  In  Job^lat*d  procurement  and 

the  types  of  help  that  thSsS  procJrem^t  SitItSS?;  *5!  Indicate 

learning  besides  the  obstacles  they  SreuSt^  in^i***''  «“*  their 

learning  projects  and  the  methoS  thS”se*to'maste?that*^J.^J* 
provide  a better  understanding  of  the  ‘^tent.*  This  research  should 

as  well  as  the  effect  of  thel?  oreserelre^CLM^S  of  procurement  practitioners 

ability  to  learn.  Preservice  education  and  experience  as  It  affects  their 

^ I ..  r.  , . . J 

Wesearch  Questions  * . M ’ • ‘ • >:-vv 
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Key  Concepts  and  Terns  | 

The  key  concepts  end  tens  are  briefly  presented  for  the  purpose  of  providing 
oons1stenc;y  as  well  as  the  criterion  for  usage.  i 

Hiny  different  terws  or  expressions  have  been  used  for  self-learning.  These  ; 

Include  self-education,  se1f-aot1vat1on,  self -teaching.  Independent  study.  Individual 
learning,  self-directed  learning,  and  self-stucly.  The  tern,  "self-learner,*  Is  used  to 
discuss  this  phenowenon  of  people  uho  engage  In  learning  activities  without  the  use  of  ] 

a teacher.  ! 

1 

This  research  Is  concerned  with  Individuals  working  In  the  field  of  federal  ( 

procureuent  at  their  job-related  self-learning  projects.  These  self-learning  projects  | 

consist  of  a series  of  episodes  totaling  seven  hours  within  a one-year  period  that 
have  the  purpose  of  not  only  gaining  a definite  knowledge  and  skill,  but  also  retaining 
that  knowledge  and  skill  for  at  least  two  dAys«(15,  16)  ^ 

RESEARCH  DESI6H  ^ \ 

. 1 

This  research  Is  an  exploration  Into  the  phenowena  of  the  practice  of  Job-related 
self-learning  projects  perfonaed  by  cIvlllaB  procureaent  personnel  In  the  federal  govern- 
■ent.  The  goal  of  this  type  of  research  Is  the  Identification  not  only  of  variables 
but  also  of  the  relationships  between  and  anong  variables. (17) 

Sources  of  Research  Data  | 

Over  72,000.'-persons  are  eaployed  In  tlie  procureaent  occupations  by  the  federal 
govenwent.  These  occupations  couprise  several  occupational  groups  coaposed  of  both 
civilian  and  arilltary  personnel. (18)  Of  this  latter  nunber,  slightly  over  3,300  are 
eipipyed  by  one  federal  agency. 

Of  this  universe  of  3,300  eaployees  In  the  agency's  procurement  organization, 

459  are  assigned  to  one  buying  center.  The  GS  grade  levels  for  these  459  persons 
range  from  GS-5  through  GS-16.  At  the  center,  GS  grades  5 through  11  are.  In  the 
procurement  occupation,  considered  to  be  training  or  entry  grades.  Persons  within  this 
grade  are,  for  the  most  part,  participating  In  a prescribed  training  program.  The 
Journeyman  and  senior  levels  are  In  grades  GS-12  and  higher.  Of  t'  * 459  procurement 
personnel  In  the  GS-1101  and  GS-1102  series  at  the  center,  340  are  ^rade  GS-12  and 
hlgher.(19} 

From  the  population  of  340  procurement  practitioners,  a sanple  size  of  at  least 
10  percent  of  the  population,  34  persons,  was  selected.  This  sample  size  was  dictated 
by  the  practical  realities  of  the  research  situation.  Selection  of  the  sample  was 
accomplished  by  a systematic  random  sample  design.  This  selection  procedure  Is  considered 
the  more  rigorous.  This  sairple  design  offers  the  advantages  of  providing  for  the  strat- 
ification of  the  population  by  GS  grade  level  and  thereby  avoids  variability  of  the 
simple  random  design. (20) 

Data  Collection 


The  structured  personal  Interview  method  Is  the  principal  means  of  collecting 
the  data  from  the  procurement  personnel  performing  Job-related  self -learning  projects. 
Additional  data  from  the  archives  complements  the  Interview  data. 
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LIST  OF  VARIABLES  AND  THEIR 


, RESPECTIVE  MEASUREMENT 
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* Variable 


’^Type  of  ’ 
Measurement 


Adult  Learner 


* ■ vO^  T; 


Total  Number  of  Projects 
Total  Number  of  Hours  Spent 
Job  Role:  Line/Staff 

Management/Operative 

Age 

■ Civil  Service  Grade/Rank 
Formal  Education  Level 
Recency  of  Formal  Education 
Longevity  of  Current  Grade/Rank 
Longevity. of  Government  Service 
Suggested  Organizational  Changes 
'Suggested  Preservice  Education 
Suggested  Preservice  Work 
Experience: 

Varieites 

Preferred 


Interval/Ordinal*' 
Interval /Ordinal* 
Nominal 
Nominal 

Interval/Ordinal* 

Ordinal 

Ordinal 

Interval/Ordinal* 
Interval /Ordinal* 
Interval /Ordi na 1 * 
Nominal 


Nominal 


Nominal 

Nominal 


Self-Learning 

Project 


Subject  Hatter 
Learning  Strategy 
Assistance  in  Deciding  to  Begin 
Project 

Source  of  Difficulty  During 
Project 

Source  of  Assistance  During 
Project 

Learning  for  Credit 
Time  Spent  on  Each  Project 
Time  Uility  of  New  Knowledge 
Status  of  Each  Project 
Job  Role:  Line/Staff 

Management/Operati ve 


Nominal 
Nomi  nal 


Nominal 


Nominal 


Nominal 
Nomi  nal 
Nominal 
Nominal 
Nominal 
Nominal 
Nominal 


Although  interval  measurement  seems  appropriate  for  these  variables,  the 
nature  of  the  data  is  such  that  the  assunptions  for  using  interval  data 
cannot  be  made,  and  ordinal  measurement  is  therefore  used. 
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Dita  Gathering  Instnnent 


u 


/ ^ ^ta  gathering  Instrumt  for  this  research,  for  the  aost  part,  replicates 

I^tl5*(2lj  *"**  ***  students.  J.  U.  Fair  and  Cr«sy 

sheet  "u^adlnSS  Intewlej^chedule.  the  probe  sheets  and  project  daU 

n Ir^  addressing  the  subject  arvas  of  the 

oTSKSSt  p.iJiS.SS'cS)*  ""I  ««»i  k,  tt.  .uff  of  tK 


Pita  Analysis 


L 


- ^ Mriables  for  which  aeasurenent  Is  obtained  are  classified  fetn  tM 

' Ei: 

SSSfSSir-fl  ~ s s.'LS7i.‘ss~. 

I / 

aoodne«'’o?^?“\!I5*t!I'*^S‘c*  **“  statistical  tests  used  are  the  Q.1  Square  test  for  ‘ 
gwj"«s  of  fit,  and  the  Chi  Square  test  of  Independence  and  the  FisherExact  ^obSlllly 

war  * primary  variable  Is  a statistically  significant 

!?  !??!!  ‘'“^POPujatlon  for  the  participation  In  self-leaming  projects  and  (2)  a 

Significant  association  exists  between  the  primary  variables  and  the  other 
fo^MlrreSOTtont^*^*  accomplished  by  using  as  the  independent  observation  tte  Sde 

RESEARCH  HNOINGS 
Saaple 

"T^sis  r“'  r “;ro?r:r ' 

r/Kisjir." ».  sf;te!SpcS:r«iS.*’“'  ^ '*“*"*  '•«'* 

Research  Question  Nuaper  1 

RQi:  How  extensive  are  self-learning  projects  practiced  among  procurement  personnel? 
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Wig^r  of  Self-Learning  Projects  Ttr?  4,^: 

« -u  job-relatetf  self-learning  o.njects  were  performed  by  32  learners 

of  ^ 34  procumwit  personnel  In  the  sa^rle.  U.e  maM:r  ranged  froa  as  few  as  tm 
^Jects  to  as  imy  as  24  projects.  The  aedian  nu  *tr  of  projects  equaled  11  for  both 

^ ***«  s««Ple  was  11.9  projects  tdille  the  aean 

ror  perfornm  was  12.7  projects.  Although  the  self-learners  are  not  evenly  distrl- 

#«*'?!!!  self^mlng  projects,  the  variation  of  the  nuBber 

See  Table  2.) 


J«St?oSS!^*{5-"*M*?'fr*  brta;'or  lVtt;Wjiatt^"pn«r4«t 


* ▼ tends  toward  a dlfferercfa!  behavior,  the  Infbmatlon  presented 

in  Table  o ^at  job  role  (both  line/staff  a-d  aanagenent/operative)  provides  a 

differential  basis  for  such  self-learning  behavior. 

,,  ^ 1 N*.  r, ifUc 

Hours  Spent  on  Self-Learning  Projects  ' i 

21  ^ pn»cure*nt  personnel  spent  nearl" 

^ nwber  of  hours  ranged  fro*  as  few  as  132 
hours  to  1,840  hours.  The  median  value  was  578  hours  for  the  32  self-learners  while  th. 
rean  value  was  655  houis.  The  variation  of  hours  spent  on  teTf-leamlng  projects  *7  the 

spent  at  self-learning  Is  a sigilficant  differential 
behavior  In  the  population  of  procurement  practitioners.  (See  Table  2.) 

infhrJl?^  V”^  on  self-learning  projects  Is  a differential  behavior,  the 

f^or*SIe*^av1or**^'*  ^ reveals  that  Joo  role  provides  a sl^ilflcant  differential  basis 

• • . I *Ir  i?  . 

Sutiwarv 

self-learning  activity  for  procurement  practitioners  is  a significent 
®®?e  engaged  In  self-learning  activity  more  than  others, 
self-learning  projects  and  spend  more  time  In  self-learning 
tend  to  be  involved  in  management  and  staff  work  rather  than  operative  and  line  work.^ 

Research  Question  Nuinber  2 

‘yP*  subject  matter  studied  by  procurement 
personnel  in  their  Job-related  self-learning  projects? 

Distribution  of  Self-Learning  Projects 

self-learning  projects  performed  are  technical/ 
business  oHented.  Over  half  are  oriented  exclusively  to  procurement  office  technical 
natters.  The  remainder  are  oriented  toward  human  and  conceptual  skills. 

Distribution  of  the  Sel f -Learners 

The  distHbution  is  concerned  with  a specific  modal  value  reported  for  each  self- 
Nearly  all  of  the  self-learners  confined  their  activity  to  the  business/techni- 
latter  being  oriented  to  procurement/technical  subjects, 
exists  for  the  procurement  categories  of  subject  matter  for  the 
Job-related  self-learning  projects  of  procurement  personnel.  (See  Table  2.) 
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DIFFEREHTIAL  E£HAVI0R  OF  THE 
^ PRIMARY  VARIABLES 
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Research 

Question 


Primary  ' ; 
Variable 
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Number  of  Projects  (Total) 
Number  of  Hours  (Total) 
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Types  of  Job  Related  Subject  Matter 

{ 

Learning  Strategy 

r" 

Assistance  in  Deciding  to  Begin  Project 
Difficulty  Encountered  During  Project 
Assistance  Received  During  Project 

f • ^ - 

Learning  for  Credit 
Organization  Changes  to  Aid  Projects 


Preservice:  (1)  Education 

(2)  Experience  (Varities) 

(3)  Experience  (Specific) 


♦Significant  at  .1 
♦♦Significant  at  .2 
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Msocittlon  with  Other  Viiiables 
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ItM  InforMtlon  In  T«b1e  4 reveals  that  at  the  lowr  sl9i1f1canoe  levels  five 
variables  tend  to  be  associated  with  subject  natter.  Concerning  Job  role,  those  self* 
learners  perfbtvlng  staff  duties  confined  their  activities  to  only  procarenent  subject 
■atter  while  those  perfonrtng  procureaent  line  duties  Included  other  subject  categories. 
Hie  tiue  utility  of  the  newly  acquired  knowledge  Is  associated  with  subj^  aatter  studied 
In  that  subjects  other  than  procureaent  have  a wch  longer  tiae  utility.  4s  to  the  tly 
period  for  each  se1f*1eam1ng  project,  projects  concerned  with  procureaent  subject  aatter 
tend  to  last  one  week  or  less,  while  projects  dealing  with  other  subjects  are  evenly 
distributed  over  different  tine  periods.  Learning  stratey  (sel f-pl anned/other)  Is  also 
associated  In  that  procurenent  subjects  were  preponderantly  self-planned,  while  other  < 
subjects  were  exclusively  self-planned.  Self-learners  dealing  with  procureaent  subjects 
had  nore  difficulty  In  their  planning  and  evaluation  of  the  projects.  . ■ > i.  . 


,v.- 


hesearth  Question  Hudier  3 

‘ Rh:  What  Is  the  nature  of  the  learning. strategy 
personnel  In  their  job-related  self-learning  projects? 

Distribution  of  Self-Learning  Projects 
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cloyed  by  procamoent 
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Over  half  of  the  405  self-learning  projects  were  acco^illshed  through  a nt»d  stra- 
tegy rather  than  one  of  several  specific  strategies.  Hie  strategies  Identified  with  self- 
learning  (one-to-one  and  self-planned)  were  e^iloyed  In  nearly  18  percent  of  the  projects. 

* * 7 ^ 

Distribution  of  the  Self-Learners  j - . 

Hie  distribution  Is  concerned  with  a specific  nodal  value  reported  for  ea^  self- 
learner.  Although  all  categories  of  learning  strategies  are  Indicated  by  the  snqr^e 
of  self-learners,  the  larger  nunher  of  self-learners  eigiloy  a nixed  learning  strategy 
idille  the  next  larger  niadier  seen  to  use  the  self-planned  strategy.  A statistically 
significant  differential  preference  exists  with  respect  to  the  learning  strategy  CBloyed. 
(See  Table  2.)  » — r 

Association  of  Learning  Strategy  with  Other  Variables 

Hie  association  of  learning  strategy  with  other  variables  Is  presented  from  the 
viewpoint  of  three  different  groupings  of  learning  strategies:  (1)  nixed  strategy  or 
the  others.  (2)  self-planned  strategy  or  the  others,  and  (3)  coeblned  one-to-one  wd 
self-pirjined  learning  strategy  or  the  other  strategies. 

Mxed  Learning  Strategy/Other  Strategy 


i 


Froa  the  Infonaatlon  In  Table  4,  a significant  association  exists  idth  tiae 
0^  newly  acquired  knowledge.  Uhen  the  utility  Is  longer  than  one  weelcTT  alxed 
strategy  as  well  as  the  other  strategies  Is  used.  At  lower  significance  levels,  additional 
differential  bases  energe.  The  type  of  assistance  encountered  during  a project  where 
the  slxed  strategy  Is  used  tends  to  occur  In  the  planning  and  evaluation  area,  whereas 
the  assistance  occurs  In  organizing  area  when  the  other  strategies  aro  used.  The  olxed 
strategy  tends  not  to  be  used  In  the  projects  having  a completed  status.  Concemlna 
Job  role,  line  procurement  personnel  tend  to  use  other  strategies. 
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Self-Planned  Learning  Strategy/Other  Strategies 
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The  Inforvatlon  In  Table  4 sfkws  that  at  lower  significance  levels,  three  associa- 
tions appear- job  role  (llne/staff),  subject  matter,  and  time  utility  of  newly  acquired 
IcfNMiedge.  Line  procurement  personnel  tend  to  use  a strategy  other  than  self -planned 
tdille  staff  procurement  personnel  tend  to  use  the  self-planned  strategy.  Self-planned 
learning  strategy  was  used  exclusively  for  subjects  other  than  procurement  while  both  • 
self-planned  and  other  learning  strategies  were  used  for  prociirement  subjects.  For  new 
bwMledge  to  be  used  for  longer  than  one  week,  strategies  other  than  self-planned  tad 
to  be  employed.  . ....  , -a 

Self-Planned  and  One-to-One  Learning  Strategies/Other  Strategies  ' 

The  Information  In  Table  4 shows  that  at  lower  significace  levels  two  associations 
appear.  . ^ = ..i >-i 

The  association  with  Job  rnle  (line  /staff)  Indicates  that  line  procuremat  person- 
nel tend  to  use  strategies  ^er  than  the  self-planned  and  one-to-one  strategies  idille 
suff  persmnel  tad  to  use  these  strategies.  ..  . 

* .-.  I-  ■ ft, -1. 

The  association  with  the  time  uttllty  of  newly  acquired  knowledge  reveals  that 
wh«  the  new  knowledge  Is  used  less  thw  one  week,  the  self-planned  and  one-to-one 
strategies  tend  to  be  used.  For  lager  periods,  other  strategies  are  used. 

Sjg-py  •:/  , . V . ; 


■‘-I  in  eir.Ir' 

The  sfxed  strategy  and  the  se1f-p1annod/one-to-one  strategies  were  not  only  the 
■lost  frequently  enployedp  but  also  tended  to  associate  with  other  variables  In  the  lower 
of  significance.  The  Job  role  (line/staff)  of  the  self-learners  provides  the 
differential  msIs  for  the  use  of  the  type  of  learning  strategy  In  that  line  procureaent 
personnel  tended  to  eiploy  other  strategics  In  addition  to  self-planned/one-to-one. 

Also,  the  time  utility  of  the  newly  acquired  knowledge  provided  another  differential  basis 
for  the  type  of  learning  strategy  used  In  that  self-planned  strategy  is  used  for  the 
shorter  periods  of  utility. 

Research  Question  Nutiiier  4 

RQ4:  What  Is  the  nature  of  assistance  received  by  procurement  personnel  In 
decidfnQ  to  bealn  .lob-related  self-1  eaminq  projects? 

Distribution  of  Self-Learning  Projects 

Assistance  In  deciding  to  begin  a project  was  received  in  slightly  over  10  percent 
of  the  projects.  The  procurement  practitioners  In  the  sanple  did  not  receive  this  kind 
of  assistance  In  90  percent  of  the  projects.  . . , 

Distribution  of  the  Self-Learner 

The  modal  value  reported  for  the  32  self-learners  reveals  a distribution  where  all 
but  one  of  the  self-learners  did  not  receive  assistance  in  deciding  to  begin  their  self- 
learning  projects. 

This  shows  that  a statistically  significant  differential  behavior  exists  In  the 
population  for  not  receiving  assistance  In  deciding  to  begin  a project.  (See  Table  2.) 
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other  vaHebles^"^""*^*"  presented  fe  Table  4,  this  vaHable  Is  Indepeadent  of  the 
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.-w  "®*  receive  assistance  In  deciding  to  begin  their 

tehavlor”*^**^  ^*^^**^"  '’®  differential  basis  for  the  prevalence  of  this  • 
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personneT  duHng  their  Job-related  self-learning  projects? 

PIstHbutlon  of  Self-teaming  Prelects 

.«»».S!3*5!i*2n  ?*  procure*nt  Job-related  self-learning  prejecte 

^ percent  of  the  projects'  sources  of  difficulty;  organizing  both 

nlM^’l^^r^^  percent  of  the  sources  of  difficulty.  Plan- 

sel  fjleaming  proj^  accounted  for  over  26  percent  of  the  sources 

Plstrt button  of  Self^Lcamers  ' - -»  ''j*  f ,r  f 

■ leamer*^air^^2?fl^M^I!I*‘*r‘^^  **’“*  reported  for  each  self- 

hlS5t  fr2J2«<yl!^  organizing  of  materials  for  the  self-learning  projects  having  the 

.A  r ' , 

sl^^caot  differential  occurrence  exists  In  the  distribution  with 
respect  to  the  sources  of  difficulty  eocountered  on  Job-related  self-learning  projects. 

tesodatlOT  of  Sources  of  Difficulty  Encountered 
wiring  Self-Learnlno  Pn^ecT 

InfornatiM  presented  In  Table  4,  five  other  variables  are  associated  with 
revealing  a differential  basis.  The  source  of  difficulty 
J*“"  one  week  occurs  In  the  organizing  of  the  materials  and*^ 

ffof  t^  Planning  and  evalua- 

in'’hn?h  Hi?  While  line  procurement  personnel  report  the  sources  of  difficulty 

reoor?ti^^so^^of"2^ff^«^'^"^"1^®«•’?^‘’®"  “‘«9®'^es»  staff  procurement  personnel 
report  the  source  of  difficulty  only  In  the  organizing  area. 

Assls^ro  In  plannlng/evaluatlon  and  organizing  occurred  directly  with  the  reoorted 

ssSd'JnJlL'Ss'x- JiTus";  V"  p'r'W”*' •» s, 

diffl^fti^I^rJ  ^ retained  longer  than  one  week.  The  source  of 

^ ^ ‘’*®  organizing  of  mateHals  and  people  when  procurement 

sublMte  *‘*!^^®“^tles  In  planning/evaluating  tended  to  occur  when, 

subjects  other  than  procurement  were  studied.  . 
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For  the  self-learning  projects,  aich  of  the  difficulty  had  as  Its  source  the 
organizing  of  nateiials.  this  differential  occurrence  uas  also  prevalent  anong  the 
Mif-leamers.  The  differential  bases  for  these  occurrences  seen  prlnarily  to  be: 

(1)  the  Job  role  of  the  self-learner,  and  (2)  the  tine  spent  on  a W-relaM  self- 
learning  project;  and  secyndly:  (1)  the  tine  utility  of  new  knowledge.  (2)  the  subject 
natter  studied,  and  (3)  the  assistance  encountered  during  a project. 
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Wesearch  Question  Winter  6 


: Vhat  is  the  nature  of  the  source  of  assistance  received  by  procurenent 
personnel  during  their  Job-related  self-learning  projects? 

Distribution  of  Self -Learning  Projects  ' ' 

The  organizing  of  naterlals  and  people  1$  a substantial  source  of  assistance  to  the 
procurenent  people  during  the  self-learning  projects.  With  these  two  categories  co^iris- 
Ing  over  81  percent  of  the  projects,  planning  as  a source  of  assistance  Is  a distant 
third  place,  being  cited  In  less  than  16  percent  of  the  projects. 

Distribution  of  the  Self-Learners 

The  distribution  ci  cemed  with  a specific  nodal  value  reported  for  each  self- 
iMmer  rewals  that,  as  with  the  self-learning  projects,  organizing  peoole  and  naterlals 
^ the  highest  frequency.  A stotlstically  significant  differential  occurrence  exists 
with  respect  to  the  sources  of  assistance  encountered  In  Job-related  self-leamlno 
projects.  ’ 

. . " ^ 

Association  of  Source  of  Assistance  Encountered 

mrlnq  Self-Learning  Project  > ■* 

For  the  Infdmatlon  presented  In  Table  4 a differential  basis  energes  with  three 
* j role--nanagenent/operat1ve;  learning  strategy  used— nixed/speclflc; 

and  difficulty  encountered).  Assistance  In  terns  of  organizing  people  and  materials  • 
occurs  when  a specific  learning  strategy  Is  used  as  opposed  to  a wlxed  strategy, 
fesistance  In  the  fom  of  organizing  people  and  naterlals  tends  to  occur  with  the  sa» 
forms  of  source  difficulty.  Assistance  In  the  font  of  organizing  people  and  naterlals 
occurs  almost  exclusively  for  operative  level  procurement  personnel  In  their  self-learn- 
ing project,  whereas  assistance  for  nanagement  level  procurement  personnel  occurs  In 
both  organizing  and  planning/evaluating.  This  Is  the  case  In  the  first  two  mudal 
gro  things. 


Sumnary  ^ 

For  both  self-learning  projects  and  the  self-learners,  the  pirlnaiy  source  of 
assistance  encountered  during  a Job-related  self-learning  project  Is  the  organizing  of 
mour^— people  and  materials.  A differential  basis  tends  to  emerge  with  variables 
for  (1)  Job  role  (management  level /operative  level).  (2)  learning  strategy  (mixed/ 
specific),  and  (3)  source  of  difficulty  encountered. 
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Researdi  Question  Huabgr  7 


4 


What  1$  the  nature  of  learning  for  credit  In  Job-re1at»d 
pnjjectsVrformed  by  procunnnent  personnil? 

Less  than  5 percent  of  the  self-Teamlng  projects  were  taken  for  credit. 
Distribution  of  Self-Learners 


>-  ("3 

nlng  projects  were  taken  for  credit.  " A 
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Iweclatlon  of  Learning  for  Credit  with  0th 


"i2Li5J^6LSSiX^.  -■ 
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Research  Question  Nuufeer  8 • 

MstHbutlon  of  Sel  f-Leanmrt  ' * ' :^  ■ •:  v ' 

Assodatlon  of  Organization  Changes  to 

Facilitate  Learning  with  other  VarTiSTes  ; - 

variables'*  ^ "®  association  exists  between  organization  changes  and  othe- 

Sunrary 

two  cate2ri«  self-learners  Is  somewhat  significant  statistically, 

^ categories  oriented  toward  resource  connitnient  account  for  over  50  oercent  of  th« 
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Il8se«rdi  Question 


' . t JiS — i 


" RQq:  What  Is  the  nature  of  the  changes  In  preurvlce  education  and  experience  , 
perceive  by  procurement  personnel  that  facilitates  job-related  self-learning? 


Preservice  Education  (Fields  of  Study) 

' The'  dIstHbutlon  of  the  34  self-learners  aaong  the  six  categories  of  fields  of 
study  does  not  reveal  a differential  preference.  (See  Table  2.) 


lA  OJUif.,,  . u-jc-riv 

- ■ - -c-r^vicV 

a i! : V-  * 


Differentiation  between  fields  of  study  basis  energes  with  total  nm*)er  of  projects. 
Those  with  a business  field  of  study  orientation  tended  to  have  uore  Job-related  self- 
leamlng  projects.  Host  of  those  In  GS  grade  levels  higher  than  GS-12  tended  to  prefer 
a business  field  of  ,*tudy.  The  business  field  of  study  tended  to  be  preferred  by  the 
older  procurement  practftlonei'S. 

• ?• 

Fresendce  Experience  (Variety)  ' 


^ - - - - 


The  distribution  of  the  34  self-learners  among  the  categories  of  variety  of  ' 
experience  reveals  a differential  preference.  (See  Table  2.)  •;  . , 4;'3.  . 

The  Information  In  Table  3 Indicates  no  d1  fferentatlon  of  presenrice  experience 
variety  with  the  other  variables.  , »,  * >» 

Preservlce  Experience  (Sped ff el  ' ' ’ ' ••  '•  • 

The  dIstHbutlon  of  the  34  self-leamers  among  the  categories  of  specific  preserv- 
ice experience  reveals  differential  preferences.  (See  Table  2.)  ' , 


The  infonnatlon  In  Table  3 indicates  a differentiation  of  specific  preservice 
experience  with  five  variables.  Those  who  preferred  a specific  preservice  experience 
tend  to  have  itore  recently  completed  their  formal  education.  A specific  type  of 
preservice  experience  to  facilitate  their  self-learning  activity  is  prcferi^  by, both 
those  with  and  without  college  degrees.  Those  in  nanagement  roles  also  prefer  a specific 
type  of  preservice  experience.  ^ ..v  ‘ j 

Suirory 

The  type  of  preservice  education  in  terras  of  fields  of  study  is  not  significant. 

To  facilitate  self-learning  activity,  however,  the  nunfcer  of  self-learning  projects  is 
significantly  different  between  preservice  education  categories  at  a level  less  than  the 
stated  level.  Age  and  formal  education  are  similarly  differentiated. 

Although  the  variety  of  preservice  experience  is  a significant  behavior  among 
self-leamers,  it  is  not  significantly  differentiated  with  other  variables. 

The  specific  type  of  preservice  experience  is  a significant  behavior  among  the 
self-leamers.  At  lower  levels  of  significance,  it  Is  differentiated  with  self-learning 
activity,  level  of  formal  education,  the  job  role  of  self-leamers,  their  age  and  recency 
of  their  formal  education. 


.*1 


Incidental  Findings 


There  are  findings  which  are  incidental  to  the  research.  First,  the  differences 
between  high  learners  and  low  learners  in  terns  of  self-learning  actfvtty  are  discussed. 
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Finally,  an  exanple  Is  one  such  project  that  was  perfonned  by  one  of  the  respondents. 
Extensiveness  of  Self-Learning  Actl^ty  J . . * ^ 


of  the  procureoent  personnel  Interviewed  engaged  In  aore  or  greater  aaounts 
of  self-learning  activity  than  others.  Those  self-learners  who  engaged  In  substantial 
self-learning  activity  are  referred  to  as  high  learners.  Conversely,  those  that  engaged 
In  alnlaal  self-learning  are  referred  to  as  low  learners.  Outing  the  course  of  the 
Interviews • sowe  Individuals  would  apologize  for  not  having  done  any  or  as  auch  self- 
learning  activity  as  they  think  others  have.  On  the  other  hand,  others  would  brag  about 
their  doing  aore  self-learning  than  others.  Regardless  of  which,  each  would  offer 
unsolicited  reasoir  why  they  do  engage  In  a lot  or  very  little  self-learning.  These 
reasons  seea  to  fall  Into  three  groups— attitude.  Interpersonal  relations  and  organiza- 
tional Identity— and  differentiate  the  high  learners  froa  the  low  learners. 

The  attitude  of  the  Mgh  learners  Is  revealed  by  their  enthuslasa  for  their  job 
and  career  field.  They  seea  to  thoroughly  enjoy  their  work  and  Its  aany  challenges. 

The  attitude  of  the  low  learner  was  reflected  by  somewhat  lower  levels  of  enthusiasm. 
Although  some  low  learners  seem  to  enjoy  their  work,  others  simply  tolerate  It.  Some 
had  less  than  five  years  of  service  remaining  until  they  mere  eligible  for  retirement. 

A few  felt  that  they  knew  all  that  was  necessary  to  know.  , 

High  learners  seea  to  possess  better  Interpersonal  skills  than  the  low  learners. 

Mai\y  of  the  low  learners  seem  to  have  difficulty  In  the  Interpersonal  relations  with 
their  peers,  superiors  and  subordinates.  Moreover,  their  level  of  aspiration  seemed 
much  lower  than  that  of  the  high  learners.  Low  learners  felt  they  would  not  be  promoted 
whereas  high  learners  aspired  to  higher  level  Jobs.  Two  of  the  high  learners  have 
recently  been  promoted. 

Finally,  the  high  learners  seem  to  be  Identified  with  the  more  prestigious  and 
publicized  organizations.  They  seem  to  work  In  Jobs  where  they  are  highly  idsible  to 
the  top  executives.  Low  learners  seem  to  be  loccted  In  the  less  prestigious  and  less 
visible  organizations.  Several  respondents  apologized  for  having  so  few  projects.  Th^ 
further  Indicated  that  In  past  periods  when  their  organization  was  the  more  prestigious 
and  publicized,  they  had  many  more  Job  demands  and  engaged  In  many  more  self-learning 
projects. 

Exaiiple  of  a Self-Learning  Project 

One  respondent  had  worked  for  several  years  In  an  office  where  the  pricing  of 
proposals  was  performed  by  the  pricing  staff.  This  Individual  was  recently  assigned  to 
an  office  where  the  pricing  of  proposals  was  performed  by  the  Incumbents  rater  than  the 
pricing  staff.  Although  this  Individual  had  had  experience  In  pricing  proposals,  he 
felt  that  he  needed  to  know  how  to  use  other  techniques.  One  such  te^lque  he  wanted 
to  learn  to  use  was  the  Improvement  or  learning  curve,  since  this  technique  had  Indlate 
application.  That  Is,  the  pricing  task  confronting  him  required  the  use  of  this  technique. 
Having  i^tlfled  idiat  he  wanted  to  learn,  he  consulted  one  of  the  supervisors  1r  the 
pricing  office  as  to  how  he  might  learn  to  use  the  technique.  The  pricing  supervisor 
first  loaned  this  impendent  a couple  of  books,  citing  the  passages  to  sludy  and  set 
up  a series  of  several  meetings  where  this  Individual  as  a learner  would  have  to  accom- 
plish several  tasks  pWor  to  the  meeting.  In  this  case.  It  Involved  using  the  technique 
on  his  work  assignments.  After  several  meetings,  which  took  place  over  a four  month 
period,  the  Individual  and  his  tutor,  the  pricing  supervisor,  were  satisfied  that  he 
possessed  sufficient  skill  to  use  the  technique  In  his  curr^t  ass1gf.*>ent. 
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IRIS  exa^>1e  Illustrates  how  an  Individual  could  conduct  a self-leanrino  Draicrr  ' 
Me  was  ready  to  learn  a specific  knowledge  and  skill.  MTenploJed  a tutoSl®o?^£fLl 
one  learning  strategy  using  an  expert.  The  Job  deaands  decided  that  he  should  beoln 
^ a learning  pro^t.  His  ealn  difficulty  was  deciding  how  Mich  to  W HU 

Si  ““  •"  He  recelvMlnodocSSIury  Credit 

Sr  of  his  own  tine  botb  on  iLl  off 

xm  job  and  with  about  tai  hous  being  spent  wf^  the  exoert  Jhm  utiiifv  a# 

•toulred  skill  was  1«>d1ate  and  will  cStlnSe  toSTgSt  th1s^^^J2l?S;2 
Sjletod!'*^'*  “**®^***  Poetical  purpose*,  the  self-toaraing  project's’ 


Conclusions 


1. 


-i;' >1 
■v-*  . 


i-V-yir^'W 
V--.  O -■‘If' 


**1^”1***'*^**®  extensive  In  the  federal  orocureaent  oocmatlons  Md  llie 
extensiveness  Is  differentiated  by  the  Individual  Job  roles. 

al*o*to  oriSted*SSii^LIIii^Sl^  preponderantly  procurenent  office  oriented,  but  t. 
also  IS  Oriented  toward  applications  of  new  as  well  as  existing  knowled^. 

4. 

5. 

6. 


7. 

8. 


• project  Is  not  an  Inherent  part  of  Job- 

related  self-learning  actlvUgr  of  procurenent  personnel.  . , 

reswrees-^terials  and  people— tends  to  be  the  niijor  source 
Of  difficulty  1*1  Job-related  self-learning  projects. 

Finding  1^1^  resouroes-HBterials  and  especially  people-tends  to  bs  the  alor 
source  of  assistance  encowtered  during  a Job-related  self-learning  project. 

Learning  for  credit  Is  nut  Inherent  to  the  self-learning  activity. 

Changes  In  organization  are  not  too  Inportant  In  Job-related  self-learning  activity. 

Kl5te51el?-“aSlnTI?«rityr"^  experience  preferences  are  Inportant  In  Job- 

Inpll cations  for  Management 

* practical  riet^int.  manageoent  Includes  those  who  are  concerned  with  1n- 
developoHit  of  procurement  practitioners.  This  Includes  not  only 
procurement  career  boards,  but  also  (1)  those  executives,  managers. 

pre^r^t  operations  within  government  agencies.  «d  (2)  the 

teachers,  writers  and  librarians  of  the  various  stowls 
and  training  organizations,  both  government  and  proprietary. 

aii<i  ^‘lucotlon  and  training  faces  a financial  crisis 

“**  ®^  Independent  study.  The  Civil  Service  Coarisslon  Is 
progra*  of  occupation-oriented  self-education  and  training  * 

^ fe‘terel  Perwnnel  Manual  stipulates  that  the  head  ?f 
s^-tSiMno  for  encouraging  the  self-education,  self-laprovewt,  and  • 

^ ^ en»)loyeo  of  the  agency.  President  Lyndon  B.  Johnson  directed  the 

aSf at^ll  S"^*®''  the  effectiveness  with  which  self-development  Is  encour- 

aged at  an  levels.  The  Government  Employees  Training  Act,  P.L.  85-507.  as  codified 
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The  views  set  forth  in  this  article  are  those  of  the  author  and  should  not  be 
coiutnied  tc  represent  the  official  position  of  the  US  Departwnt  o.  the  Aray 
unless  so  stated. 


•IKTROOUCTION 


Each  year  the  IMRCON  procureaent  coamunity  awards  billions  of  dollars 
( , for  everythinr  froa  housekeeping  services  to  the  research/developoent  and  ' 

production  of  major  weapons  systems.  There  is  an  obvious  need  for  well 
trained,  competent  procurement  professionals  if  these  funds  are  to  be  prop-  ' 

/ , . erly  and  prudently  awarded  and  managed.  The  research  effort  showed  that 

V.  the  DARCOH  Procureaent  Intern  Program  is  meeting  this  need. 


RESEARCH  APPROACH  . 


The  objectives  of  the  research  effort  were: 


V • I ' iv'S!  1 t ‘.;» 


t 1.  Evaluate  the  current  DARCON  procurement  intern  program  to  determine 

how  well  the  program  is  providing  graduate  interns  to  function  as  procure- 
ment professionals  for  estimated  vacancies.  ? 

2.  Develop  recoamendations  for  improvement  of  estimating  DARCOH  needs, 
recruitment,  preparation  and  retention  of  personnel  for  a career  in  procure- 
ment. 

Some  of  the  specific  questions  to  be  explored  in  the  project  were: 

1.  Is  the  training  given  to  interns  effective?  That  is,  is  the  graduate 
intern  capable  of  functioning  as  a procurement  professional? 


2.  Is  the  program  long  enough? 

3.  Is  classroom  training  being  coordinated  with  intern's  on-the-job 
training  assignments? 

4.  . Is  there  a need  for  sequencing  of  training? 

5.  What  career  management  actions  are  necessary  or  desirable  to  enhance, 
the  procurement  intern  program? 


• .4 

In  addition  to  the  traditional  literature  search,  a questionnaire  was 
distributed  to  244  OARCOM  procurement  interns  and  graduate  DARCOM  procure- 
ment interns  at  the  major  subordinate  conmands  (HSC's).  There  were  28 
specific  questions  and  two  questions  which  allot^  for  general  connents. 

The  questionnaires  were  distributed  with  self-addressed  envelopes  to  insure 
anonymity  of  replies.  The  responses  to  the. questionnaires  were  su^ected  to 
both  computer  analysis  and  manual  analysis. 

Approximately  100  structured  Interviews  were  conducted.  The  majority  of 
interviews  were  with  procurenent  managers  and  PCO's  at  the  major  subordinate 
CMunds.  Individuals  in  Civilian  Personnel  Offices  and  in  Recmitment 
activities  were  also  interviewed.  Regression  analysis  and  personnel  flow 
modeling  were  explored  as  possible  methods  of  estimating  intern  intake  i. 

WHT  A PWiaiiiEMiiT  iHTniii  rMGBAH? 

The  Amy  Procurement  workforce  is  conposed  of  roughly  3,500  individuals, 
the  majority  of  whom  (90.1  percent)  are  in  the  following  job  series: 

. 1101  General  Business  and  Industry  • ; f j'-t**,  s, 

' « ' ■ ■ c-''  ' X r. i 

1102  Contracting  and  Procurement  -- 


..  j 

U 

i 

O 

7 1 

u 

'9 

•-■K  ..--j 


1103  Industrial  Property  Adninistration 
1150  Industrial  Specialist 


''■)  ii.  jfiT  - 


The  bulk  of  the  workforce,  73.4  percent  are  in  the  1102  Contracting  and  Pro- 
curement series.  The  distribution  of  the  workforce  by  grade  is  depicted  In 
Figure  1.  For  each  grade  level  the  number  of  individuals  is  graphically 
portrayed  by  the  width  of  the  diagram.  Forty  three  point  three  (43.3) 
percent  of  the  workforce  is  concentrated  in  grades  9 through  11.  Forty  six 
point  eight  (46.8)  percent  are  in  grades  12* through  15.  . 

• • ¥ 

In  December  of  1975,  60  percent  of  the  individuals  in  grades  GS12-15 
were  over  age  50.  Table  I presents  a distribution  of  these  individuals. 
Thirty  five  point  four  (35.4)  percent  of  the  individuals  in  grades  6S12-15 
will  be  eligible  for  retirement  by  December  of  1978.  Table  II  portrays  the 
distribution  by  grade.  - • ' 

While  one  must  be  concerned  with  eventually  replacing  these  individuals, 
it  is  of  equal  concern  that  their  skills  and  expertise  be  utilized  to  train 
a replacement  workforce  to  the  same  degree  of  professionalism.  Since  the 
preponderance  of  the  Amy's  procurement  workforce  resides  in  the  OARCOH  com- 
munity, it  is  critical  that  OARCOM  maintain  a viable  means  of  developing 
procurenent  professionals  with  the  requisite  talent  to  accomplish  its  mission. 
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6S9/10:  723  individuals 


6S7/8:  197  individuals 


GS5/6:  92  Individuals 


DISTRIBUTION  OF  ARMY  PROCUREMENT  WORKFORCE  BY  GRADE 

FIGURE  1 
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HOT  IS  THE  OMMrOH  PMCMRBCnr  iWBBI  PMBMjlT 

IIITRODUCnON 


The  (WRCOH  Procuranent  Career  Intern  Progran  Is  a coaprehensive  training 


program  which  prepares  selected  Individuals  for  Journeyman  level  perforwnce 
In  the  1101 t 1102g  1103e  or  1150  Job  series.  It, provides  the  background 
for  subsequent  progression  Into  middle  and  upper  level  positions  In  the 
procurement  career  field.  . -ol> 

% 

Interns  are,  for  the  most  part,  recruited  through  the  Professional  and 
Administrative  Career  Examination  (PACE).  They  are  started  at  the  6S-5  level 
and  after  a year  are  promoted  to  6S-7.  For  their  third  year  of  training 
they  are  transferred  to  a permanent  duty  location  (PDL)  where  they  receive. 

• 6S-9.  This  structured  3-year  development  program  consists  of: 

e 

1.  Formal  school  training  which  Is  accomplished  at  designated  DOO 

schools.  Normally,  the  training  consists  of  formal  classroom  instruction,  a 
These  formal  training  courses  are  distributed  throughout  all  three  years  of 
the  Intern  program.  a . , 

• . 

2.  First  and  second  year  on-the-job  training  (OJT)  consists  of  In-depth 
rotational  training  and  Is  accomplished  at  the  career  Intern's  designated 
training  site  - one  of  DARCOH's  major  subordinate  connands. 


3.  Third  year  specialized  OJT  consists  of  Intensive  training  accom- 
plished at  the  OARCOK  career  Intern's  PDL  and  may  Include  rotational  assign- 
ments. 

4.  Successful  canpletlon  of  each  phase  Is  a prerequisite  to  connence- 
■ent  of  the  next  phase  and  to  advanceaient  to  higher  grade  levels. 

' . 

Ihirlng  the  first  and  second  years,  over  3,000  hours  are  devoted  to 
structured  rotational  assignments  and  140  hours  are  devoted  to  formal  schools 
training.  In  the  third  year  specialized  technical  training  assignments 
account  for  some  1,500  hours,  and  formal  schools  training  account  for  a 
minimum  (jf  192.  ** 

RECRUlTME)rT 

The  MRCOM  Procurement  Intern  Program  Is  centrally  managed  and  funded. 

HQ  OARCON  maintains  a pool  of  spaces  against  which  Interns  are  slotted 
regardles-s  of  their  physical  location.  HQ  DARCOM  provides  funds  for  the 
pay  of  tte  Intern,  his  benefits,  TOY  travel  and  per  diem,  and  authorized 
permanent)  change  of  station  allowances.  The  centralized  concept  eliminates 
the  responsibility  of  local  agencies  having  to  provide  manpower  spaces  and 
funds  forj  the  career  Intern.  It  also  eliminates  the  possibility  of  Interns 
being  sut^'ected  to  local  reductions-in-force. 
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sis 

■ake  firm  offers  of  enployment  to  interested  and  qualified  personnel  ” 


:}' , ^ understand  and  interpret  cooplex  readinq  naterial 

'•'-‘s  ^ use  language  where  precise  correspondence  of  words  * 

pSssibu!'*  * elective  oral  and  written  cooninication 


“II, 


III. 


decisions  or  take  action  in  the  absence  of 
problems  by  inferring 
conclusion  ^ ^ logical  ^ 

discover  underlying  relations  or  analogies  aoiong 

» fSd  problems  involves  informatior  ® 

.» and  testing  of  hypotheses.  , ^ --Kwr 


■•-i- 


IV. 


Ability  to  discover  implications  of  facts;  to  reason  from 

dweloptna 

These  five  areas  are  scored  independently  with  var'lous  weiahtfnnc  anri 

S5:£"~^ 

tional  categories  are  as  follows:  ^ 

. Category  A:  Personnel  Administration.  Management  Analysis. 

»cial  Insurance  Administration.  General  and 
^'"'^^t^Qotion.  Customs  Inspection. 

P^lic  Health  Program  Management.  Veteran's 
A^inistration  (VA)  and  Social  Security  Ad- 
■^histration  (SSA)  Claims  Examining  and 
Public  Information. 
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Category  B: 


Economics.  Psychology,  Taxation,  Budget  Ad- 
ministration, CONTRACTING  MD  PURCHASING, 
and  Financial  Institution  Examining. 


Category  C: 
Category  0: 


Automated  Data  Processing. 


Category  E: 
Category  F: 


Contact  Representative  Position's  (mainly 
with  tte  VA  and  SSA).  ^ . 

Uriting  and  Editing. 


.'.U 


I 1. . 


Alcohol,  Tobacco,  and  Firearms  Inspection. 


The  FPO's  contact  an  average  of  11  individuals  for  each  intern  accepted 
into  the  program.  About  1/4  of  these  individuals  are- interviewed  and  only 
about  1/3  of  those  interviewed  are  hired.  The  PACE  scores  of  accepted 
•interns  generally  range  from  95  to  110  and  average  approximately  98  tc  99. 


LEARNING  BY  DOING  - OJT 


.-i. 


The  OARCOH  Procurement  Intern  Program  stresses  "hands-on  learning." 
During  the  first  two  years,  interns  aie given  specific  amounts  of  on-job 
rotational  assignments  in  eight  different  areas  (see  Table  III).  These 
learning  by  doing  assigments  ensure  that  the  interns  have  ectual  work 
experience  in  all  the  inportant  aspects  of  procurement. 


During  the  Small  Purchases  rotational  assignment,  the  intern  learns 
when  to  use  oral  or  written  solicitation.  He  determines  and  recomnends 
appropriate  award  of  solicitations  and  he  gets  experience  in  the  administra- 
tion of  those  awards.  . r 


The  Formal  Advertising  OJT  consists  of  480  hours.  In  this  segiimnt  the 
•intern  is  involved  with  all  facets  of  formal  advertising  from  determining 
the  adequacy  and  completeness  of  a procurement  package  through  award  of  the 
Invitation  for  Bid  (IFB).  He  learns  how  to  establish,  review,  and  maintain 
a list  of  sources  of  supply;  how  to  preparer  the  IFB;  and  how  to  conduct  a 
bid  opening.  He  gains  experience  in  the  evaluation  of  bids,  the  handling 
of  late  bids  and  mistakes  in  bids,  and  the  difference  between  responsiveness 
and  responsibility.  He  is  also  exposed  to  the  procedures  involved  in 
protests  of awards  and  the  techniques  of  Two-Step  Formal  Advertising. 


The  intern  completes  a total  of  320  hours  of  OJT  in  the  Contract  Pricing 
area.  In  this  phase  of  his  OJT,  the  intern  learns  the  difference  betwwn, 
and  how  to  perform,  price  analysis  and  cost  analysis.  He  becomes  proficient 
in  the  use  of  the  Weighted  Guidelines  technique  to  develop  a profit  objec- 
tive and,  where  possible,  participates  as  a member  of  a Should  Cost  Team. 

The  intern  learns  the  importance  and  interfaces  of  the  technical  evaluation, 
the  audit  report,  the  Independent  Government  Cost  Estimate,  and  contract 
pricing. 
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TABLE  III 


ROTATIWWL  TRAINING  ASSIGWENTS  FIRST 

& SECOND  YEAR 

TOPIC  * . 

, . ..  • TRAINIMC  HOURS 

^Small  Purchases 

• •% « 

71 

320‘ 

' Formal  Advertising 

■ ■ ' 5 ■’  V 

’ ~ 1 

' 480 

' o<  Contract  Pricing  » 

r • • ^ 

'320 

t ■**  - • r 

. r*  H690ti#tion  ^ ^ 

!\a  • * •' 

' .640  . 

^ Contract  Management 

j 

• yX'  1 

- ;•  ' 

480 

C<f 
•'  '1. 

Procurement  Policy 

t 

/ 

320 

0m^..  . . rr* 

Production  Management 

240 

Related  Procurement  Subjects 

240 

■ 0:  ■ 4:  - 

spends  a total  of  640 
®”  P'***®*  contract  negotiation  proce- 
rilS  LT  T ^ ^ ^curement  Work  Directive  through  contract 

familiar  with  the  various  types  of  negotiation 
■ tHH**?*^*^  *S^  ^^®  P'*®P®*'®t^on  of  determinations  and  finding^  He  prepares 
and  IssMS  Requests  for  Quotations  and  Requests  for  Proposals  and  EegStiltes 

H?Dlrti?iL2r?!I^L?h  “ ^ ^'’®  contract  execution. 

oJeSlil  noncompetitive  negotiations  and 

prepares  the  additional  justification  for  noncompetitive  negotiations. 

hours  in  the  Contract  Management  area,  the  intern  is 
a^inistration  from  the  time  of  award 
^ through  final  disposition  of  the  completed  contract  file. 

S ® responsibilities  of  the  Goverranent  and  contractor 

contract  management  functions  and  respon- 
TCO.  He  prepares  and  learns  the  diffIJSSJes 
suppl cental  agreements,  and  processes  requests 
I®.  ®5^jstance.  He  becomes  involved  with  property  administration 

as^efaul?^°"^*  disputes/appeals,  and  terminations  for  convenience  as  well  * 

„n.,,^"  V*®  Procurement  Policy  area,  the  intern  participates  in  the  develoo- 
repirtSg^pKSsr*^'”"  of  policy  and  participates  in  the  procurement 
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to  the  Production  Managenent  area.  . 
purpose  and  objectives  of  mobilization  planning 
^J!?'  ^“Pocts  on  cost,  delivery,  and  procurement  . 

’*^th  Haterial  Allocation,  Bills  of  Haterial , ^ - 

i!!^4lS!!^'*ki!l3®"^#  List,  and  the  Army  Materiel  Plan.  He  performs  the 
♦4!!®“*  111!^^°***  expediting  delivery  and  learns  the' role  of  the  produc-  • * 

tion  member  on  a Pre-Award  Survey  Team.  ^ 

'’®“'^  *1:?  to  Related  Procurement  Subjects.  ' 

focuses  on  the  Interrelationship  of  Materlle  Hanaoenent.  * 

Cnaln^r^n®**r"L®®_*_m^"u®®*®®®*  C^troller,  Automatic  Data  Processing, 

Comaodity  Itenagement,  Data  Management,  and  Project  Hanagmnt 
with  the  Procurement  function. 

,1"***^  the  intern  program,  effort  is  concentrated  on  in-  ‘ 

depth  technical  on-tbe-Job  training  in  one  of  the  four  following  Job  series: 

General  Business  and  Industry  (6S-1101)  i ^2^' 

> * ■ • • ■•■■■.■  I' 

' Contract  and  Procurement  (GS-1 102)  2^^ 

, Industrial  Property  (GS-l 103)  - >. 

Industrial  Specialist  (GS-1 150) 

Depending  upon  the  target  Journeyman  assignment,  1,528  hours  are  devoted 
to  sp^ializ^  training  and  192  hours  are  devoted  to  formal  classroom 
toaining.  This  specialized  technical  training  is  designed  so  that  the  intern 
will  be  capable  of  performing  as  a professional  at  its  conclusion. 

FORMAL  SCHOOLS  TRAIHiNG  ' ^ ‘ * 

• ^ 

While  the  DARCOM  intern  program  concentrates  on  frjT,  it  does  not  ignore 
formal  schools  training.  Between  888  hours  and  632  hours  are  devoted  to 
classroom  training  depending  on  the  target  Journeyman  assignement.  A 
minimum  of  eight  courses  are  completed  during  the  three  years  of  the  program 
Upon  completion  of  the  program,  the  intern  Jias  attended  all  mandatory  courses 
for  tto  entry  level  (GS-5  through  8)  and  the  majority  of  mandatory  courses 
for  the  Jnte*^iate  level  (GS-9  through  12)  in  the  procurement  career  pro- 
gram. The  courses  are  time  phased  throughout  the  program  so  that  the  intern 
receives! maximum  benefit  from  the  courses.  In  addition  to  the  core  courses 
which  alp  interns  take,  there  are  specialized  courses  for  those  going  into 
certain  aourneyman  assignments.  Those  entering  the  GS-1150  series  attend 
the  Production  Management  course;  those  entering  the  GS-1 103  series  attend 
the  Induptrial  Property  Administration  course;  and  those  who  are  scheduled 
to  become  GS-1102  Price  Analysts  attend  the  Introductory  Quantitiative  Analy- 
sis  course.  Table  IV  depicts  the  fo.mal  schools  training. 
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♦-.  -'  , t^>J-  <K,.4>j:i.4)  * .formal  SCHOOLS  TRAINING  .,„ 
COURSE 


. ' \ , ■ ■:>.:.^-^-^^  ^^-TI^INING^HO^  : ’ 

WACOM  Orientation  Course  for  Career  Interns.  ^^4-' - Win  - -'^-0^^ 
(First  120r days)  - 

•«4r«>^S4-yH-*  •<-  ♦<•*1 


: cCr  £••  /.;f ..  Ittjy  .''-t-ihi jt  .> 

’•■sr  y-V.AA  !>.ib  .y  - . 'll 


Principles  of  Contract  Pricing. 
(Second  year) 


: - t ' I Iz  *80  p<:i  .H  j 


*' 


Defense  Contract  Negotiation  Workshop.  •*.  .^n  x y-H 

(Second  year)  .rvr  il-.w^rVrXj^,  . (J 


Automatic  Data  Processing  Orientation  Seminar. 
(Second  year) 


Contract  Adtalnistratlon  Course. 
(All  third  year) 


• ,->•  :..-!i  40  - .•'>' 

U'>V 

I'  r 

>/./  f>  ii;  120 


'V.t  .V7a  ■ ^rtS!  : 


•'  ’■:  iiVv  'y  *>^1;  ,*??«(• 


G 


Government  Contract  Law. 
(All  third  year) 


Production  Management  Course. 
(GS-1150  only) 


* '72  ~ 

■r  -•;«•.  ',  : it  '■:  • 


Introductory  Quantitative  Analysis. 
(GS-n02  Pricing  only) 


> 256 

' t#  it*  ^ 


*-•*-  ' <>  r * r: 


Industrial  Property  Administration. 
(6S-1103  only) 
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HOW  MANY  INTERNS  ARF  NFFhFn? 


*r/;  ^ . 


I 


Procurement  Interns  are  one  source  of  the  manpower  required  to  meet 
expanding  workload  requirements  or  to  replace  those  procurement  personnel 

retirement,  resignation,  death,  or  transfer  to 
another  organization.  In  an  attempt  to  forecast  their  procurnnent  intern 
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retirements » DARCOH  conducts  an  annual  survey  throughout  the  field  rr>iTvn;irv4c 
DAtOM  where  It  is  reviewed  and  tabulated  to  arrive  at  the  total  nunhpr  nf 

toifSJI'Sf  • '"“™  rew)r«nts  oS  t““ 

nM5ed^Tk2Jif!l~*'.'3  J5'  '’“J*’  *"  ““SS  of  the  actual  niober  ^ 

' 5rnfliawi^<2  !i*i  approach  to  intern  requirements  forecasting 

is  needed  which  will  provide  more  accurate  projections. 

••  *1  ! 'i.-er-,-- 

Two  apprMches  that  appear  to  have  good  potential  are  beino  invA<tinaf«fi. 

and  regression.  Regression  equations  will  be  devel-’ 
0^  if  an  adequate  historical  data  base  can  be  located.  The  imnediate 
effort  involves  designing  and  programming  a personnel  flow  model. 

eal  actual  personnel  data  currently  available  from  the  Civilian  Per- 
System  (CIVPERSINS)  as  input,  this  personnel  flow  model 
annual  pr^rement  personnel  requirements,  of  which  interns 

iSel  5?lJhiSlli!?  ^ 2 describes  the 

in  ^ designed,  gains  to  a procurement  personnel  pool  will  be 

^ intern  or  all  others.  Losses  from  the^ol  will  also 

expJISeS  la^r^^dlsiJ^S!'^"'"  detail  can 

follows.  Beginning  with  the  actual 

wirs  personnel  in  OARCOM.  probable  losses  during  the 

mirS5JIt^?!tes  The?*?i«n  9®"®™^®;. ‘^sed  upon  Civil  Service  Coranission 
r "•  losses  due  to  other  reasons  are  generated  using  loss 
history  in  CIVPERSINS.  Some  members  of  the’ 
thS  JJSftS  CIVPERSINS  promotion  rates.  Vacancies 

V ® 50™®  combination  of  intern  gains  and  other 

t'’®  ‘‘®5ired  manpower  level. 

year^des^^*^^^  printed  and  the  cycle  is  repeated  for  each  forecast 

!S  THE  PROCUREHENT  INTERN  PROGRAM  DOING  1TS>J0B? 

DATA  BASt 

- a _ 

A total  of  244  questionnaires  were  distributed.  Two  hundred  two  responses 
were  received,  out  of  which  12  questionnaires  were  rejected  for  various 

consists  of  190  respondents. 

The  majority  (40  percent)  of  those  responding  to  the  questionnaire  were 
tSiHinf  however,  there  was  a good  sampling  of  all  stages  of 

S interns  each  accounted  for  approximately 

the  respondents  and  graduate  interns  accounted  for  some  16 
^rcent.  Th?  grades  of  the  respondents  ranged  from  GS-5  to  GS-13.  Informa- 
tion garnered  from  approximately  100  structured  interview  was  also  utilized 
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INTERN  PROFILE 


i V 


The  Majority  of  DARCOH  Procurement  Interns  entered  the  program  through 
the  FSEE  or  PACE  examination.  Only  3.16  percent  entered  the  program 
through  Internal  placement  techniques.  Approximately  70  percent  of  the 
Interns  are  nonveterans  who  have  generally  entered  the  program  soon  after 
completing  college.  Some  57  percent  entered  the,  program  between  the  age 
of  20  and  25.  Few.  only  2.11  percent,  were  over  35  years  of  age  when  they 
entered  the  program. 


r 


The  educational  level  of  the  respondents  Is  very  high  when  contrasted 
with  the  educational  level  of  the  Amor's  civilian  procurement  workforce. 

Ninety  two  point  eleven  (92.11)  percent  of  the  Interns  have  at  least  a 
Bachelor's  degree  compared  with  44.48  percent  of  the  6S12-GS15  procurement 
careerists.  Table  V presents  the  various  educational  levels  of  procureaKnt 
interns  and  the  educational  levels  of  the  GS12-6S15  procurement  professionals. 
Few  Interns  have  advanced  their  educational  level  since  entering  the  program. 
t)n1y  15  Individuals  have  moved  to  a higher  level.  Ten  of  these  Individuals 
have  completed  a Master's  degree  since  entering  the  program.  The  educational 
background  of  the  Interns  Is  quite  varied.  The  majority  majored  In  one  of 
the  Social  Sciences,  few  have  had  any  appreciable  education  In  Engineering 
or  Law.  Surprisingly,  52.63  percent  have  had  less  than  nine  semester/ 
quarter  hours  of  college  credit  In  Business  and  Comaerce  subject  areas. 


VARIETY  OF  OJT 


Three  questions  were  designed  to  ascertain  the  amount  and  variety  of 
procurement  experience  an  Intern  receives. 


The  first  question  measured  the  various  types  and  dollar  values  of 
contracts  the  Interns  had  written.  As  Table  VI  shows,  there  was  a wide 
variety  of  contract  types  and  dollar  values:  The  experience  the  Interns 
are  receiving  Is  excellent  compared  with  the  mix  of  contracts  In  the  Depart- 
Bient  of  the  Army.  In  fact.  Table  VII  clearly  shows  that  they  are  exposed 
to  more  types  of  contracts  than  one  would  normally  expect  given  the  avail- 
able mix  of  contracts.  The  average  number  of  different  types  of  contracts 
and  dollar  values  was  3.25.  This  average  W&s  distributed  by  stage  of  train- 
ing as  follows: 


First  year 


1.90 


Third  year 


3.88 


Second  year 


2.63 


Graduate 


5.10 


The  second  question  measured  the  different  types  of  procurement  actions 
the  Interns  had  accomplished.  It  was  concerned  with  determining  the  Intern's 
exposure  to  various  procurement  methods  such  as:  Small  Purchases,  Formal 
Advertising,  Two-Step  Formal  Advertising,  Competitive  Negotiation  and  Sole 
Source  Negotiation.  In  all  but  three  areas  - Two-Step  Formal  Advertising, 
Off-Shore  Procurements,  and  8(a)  contracts  - more  than  35  percent  of  the 
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EDUCATIONAL  LEVEL:  ARMY  PROCUREMENT  CAREERISTS  GS12  - 6S15 
COMPARED  TO  OARCOM  PROCUREMENT  INTERNS 
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inPES  AND  DOLLAR  VALUES  OF  CONTRACTS  -1 

_ 4,;/- MVS  .;•..*  .tiv>  .<«»  WSSlu 

’ ■-  *'■  Question  II  ‘ "i 


TYPE/S 


FP  w/EPA  ■••  ■•  43  -».  N 94  71 


FFP  $10,000  to  99,999 
FFP  1100.000  to  999.999 
FFP  over  $1 ,000.000 
FPI  $10,000  to  99,999 
FPI  $100,000  to  999.999 
FPI  over  $1 ,000.000 
CPFF  $10,000  to  99,999 
CPFF  $100,000  to  999,999 
CPFF  over  $1 .000,000 
CPIF  $10,000  to  99,999 
CPIF  $100,000  to  999.999 
CPIF  over  $1,000,000 
CPAF 
T4H 
LH 
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Interns  have  had  sow  experience.  Again,  the  wide  breadth  of  experience 
1$  evident  (see  Table  VIII).  The  overall  average  number  of  different  types 
Ktions  was  4.89.  This  avaraga  was  distributed  by  staoe  of  training  as 


First  year  * 2.52 
Second  year  4.50 


Third  year  ' 6.45 


Graduate 


6.81 


The  thlrS  question  assessed  the  Intern's  Involvement  with  actions  such 
as  Terminations.  Protests.  Claims,  and  Mistakes  In  Bids.  Again,  the  breadth 
of  experience  Is  extensive  (see  Table  IX).  The  average  number  of  different 
types  of  actions  Is  4.85  which  was  distributed  as  follows:  • . 


First  year  2.42 

Second  year  4.24 


Third  year 
Graduate 


5.90 


6.81 . 


The  responses  to  these  three  questions  clearly  show  that,  on  the  whole, 
the  Interns  are  being  given  broad  OJT  in  all  important  aspects  of  contracting. 

Further  analysis  of  this  data  revealed  a distinct  relationship  between 
the  variety  of  experience  and  how  well  the  Intern  rated  the  program  In  pre- 
paring one  for  journeyman  level  performance.  Those  who  had  rated  the  pro- 
gram excellent  had  a significantly  greater  variety  of  experience  than  those 
who  rated  the  program  poor  (see  Figure  3).  . • 

THE  CLASSROOM 

Interns  are  receiving  the  required  fonoal  training.  A total  of  98.95 
percent  have  either  completed  the  Defense  Procurement  Management  Course  or 
are  scheduled  to  attend  the  course  within  six  months.  With  the  exception 
of  Government  Contract  Law  (49.47  percent),  at  least  70  percent  of  the  In- 
terns have  either  completed  or  are  scheduled  to  attend  the  other  core  pro- 
curement courses.  The  majority  of  the  courses  were  classroom  as  opposed 
to  correspondence  courses.  (Kit  of  the  768  mandatory  procurement  courses 
which  were  completed  only  26.04  percent  were  taken  by  correspondence.  If 
the  Small  Purchases  Course,  which  Is  almost  exclusively  offered  by  corre- 
spondence. Is  eliminated,  only  12.5  percent  of  the  courses  were  taken  by 
correspondence. 

Several  courses  were  Identified  by  the  respondents  as  being  most  bene-  ‘ 
ficlal  In  day-to-day  procurement  activities.  The  Defense  Procurement 
Management  Course  was  designated  as  most  beneficial  by  140  Individuals. 

This  was  followed  by  53  individuals  who  designated  either  the  Defense  Cost 
and  Price  Analysis  or  the  Defense  Negotiation  Techniques  Course. 

The  questionnaire  asked  that  courses  be  Identified  for  addition  to  or 
deletion  from  the  Intern  program.  Thirty  one  point  ninety  one  (31.91) 
percent  suggested  additional  courses  were  needed.  Some  32  specific  courses 
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Two>Step  Fonral  Advertising 
Partial  ^et-AsIdes 

Coapetitive  Negotiation  over  $100,000 
Price/Co >t  Analysis  $10,000  to  $100,000 
Prlce/Co>t  Analysis  over  $100,00 
Sole  Source  Negotiations  over  $100,000 
8(a)  Contracts 
Off-Shori  Procurements 

Small  Purchases 

* 

Formal  Advertising 

Basic  Order  Agreement  .. 
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ACTIONS  ACCOKPLISHED 
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Protests  to  the  General  Accounting  Office 

Appeals  to  the  Amed  Services  Board  of  /: 
Contract  Appeals  : . 

T-".  - "'i.% 

Claims  Against  the  Gcvernment 

' ••  .,r 

Terminations  for  Default 

‘ i.C 

Terminations  for  Convenience 

* jt  - ‘ 

Ho  Cost  Terminations  __  ^ . 

Certificate  of  Competency  Procedures 
Mistakes  in  Bids  * ‘ * 


28  16.67., 

30 

53  . 31.55  .;V 

65  38.69  • ; . -Vi 

78  ‘ 46.43  L.“^ 


Late  Bids  or  Proposals 


Reprocurement  after  Terminations 

> Contracts  with  Government  Furnished 
Property 
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were  identified,  however,  no  one  course  was  recomnended  by 

* si^ificant  najority.  Eighteen  point  seventy-two  (18.72)  percent  of  the 
saaiple  indicated  need  to  delete  courses  from  the  program.  The  most  often 
mention^  course  for  deletion  was  the  AOP  seminar  (10  respondents),  the  next 
mst  often  rw^ended  for  deletion  was  the  Contract  Administration  Course 

- Several  of  those  recomnending  deletion  of  the  Contract 

Coulee  indicat^  that  it  was,  to  a large  extent,  a repetition 

validity,* however.  Contract  Adnin- 

istration  Is  a third  year  course  and  OPMC  Is  taken  in  the  first  year  of  the 
review  of  SOTO  subject  matter  two  years  after  comp1et1i.g 
teneficial.  Additionally.  27  respondents  indicated 
tliat  Contract  Adninistratlon  was  the  aost  beneficial  course  they  had  taken. 

m mam  mmm  ciAssRoaf  jiia  pn  / ' ^ 

intern  program  is  succeeding  in  translating  what  is  learned  in  the 
useful,  practical  work  experiences.  Seventy-two  point  zero 
answering  believe  that  classroom  training  was 
directly  related  to  OJT  assignments  received  within  3 months  after  completing 

* Only  28.65  ^rcmt  felt  that  they  had  taken  courses  which  were 

unrelated  to  subsequent  OJT  assignments.  , > 

The  majority  of  procurement  professionals  Interviewed  indicated  that 
they  make  a definite  effort  to  give  interns  OJT  assigmnents  that  relate  to 
courees  recently  completed,  and  several  stated  that  they  often  discuss  course 
content  and  how  It  relates  to  new  jobs  the  Interns  will  be  given. 

• * 

ftiT  ^ o''®?f  I?®”'  ® problem  linking  formal  training  to 

WT.  The  Defense  Small  Purchases  course  is  to  be  taken  within  the  first 
120  days  of  the  program.  This  course  is  for  the  most  part  taken  in  the 
^**^^puudence  mode.  Often  the  intern  has  completed  his  320  hours  of  Small 
Purcrases  OJT  before  he  completes  the  correspondence  course.  The  second 
problem  area  concerned  scheduling  of  the  rotational  Pricing  assigraaent  to 
occur  soon  after  an  intern  has  completed  the  Principles  of  Contract 
Pricing  course.  This  is  not  always  done. 

The  ^unt  of  classroom  training  an  intern  receives  is  related  to  how 
TOll  he  perceives  the  program  as  preparing  one  for  Journeyman  level  per- 
forroncei  Those  who  rated  the  program  -excellent"  had  attended  an  average 
of  4.26  courses.  Those  who  rated  the  program  "poor"  had  attended  an  average 

* courses.  Those  who  rated  the  program  "excellent"  were  also  scheduled 
to  attend  more  courses  than  those  who  rated  the  program  "poor." 

APPRAISING  IffTERN’S  PERFORMAHCE 

As  **pth  any  other  employee,  the  intern's  performance  must  be  periodi- 
cally evaluated.  This  evaluation  is  accomplished  semiannually  through 
tTO  use  pf  OARCOM  Form  2002,  Career  Intern  Progress  Report.  The  purpose 
of  the  report  is  twofold,  first  to  evaluate  performance,  and  second  to 
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ensure  that  the  Interns  are  receiving  the  necessary  training  assignments. 

This  report  requires  that  the  number  of  hours  devoted  to  each  training 
subject  be  listed  and  performance  evaluated  on  a. scale  of  one  to  three:  ‘ ^ 
one  being  unsatisfactory;  two.  satisfactory;  and  three,  outstanding.  The 
report  provides  for  written  coKvnts  by  both  the  Intern  and  the  Intern's 
supervisor.  i -t  v ;-i 

’ ■ • • • • - . '»* 

Fifty-five  point  twenty-five  (55.25)  percent  of  the  Interns  believe  that 
the  appraisal  form  and  procedures  provide  for  a free  and  unbiased  appraisal. 
Approximately  36  percent  of  the  Interns  recommended  changes  to  either  the 
report  Itself  or  the  procedures  Involved.  Among  the  most  frequent  were  . , . 
the  following:  . . . _ , . \ 


•A  wider  scale  is  needed  In  rating  performance."  (T7  Individuals) 

•The  Intern's  comments  should  be  sutnltted  separately  from  the  super 
visor's  coonents.*  (11  Individuals) 


"There  Is  a need  for  specific  ^.'Idellnes  In  assigning  ratings." 
(8  Individuals)  ’ 


"The  number  of  liours  an  Intern  devotes  to  a training  area  should  not 
be  listed."  (8  Individuals) 


Several  coonents  were  diametrically  opposed.  For  Instance,  some 
Individuals  stated  that  there  Is  no  need  for  a coonents  section  In  the 
evaluation,  while  others  stated  that  there  should  be  only  a coonents  section, 
toe  Individual  Indicated  that  the  grades  received  In  formal  schools  training 
should  be  noted  on  the  report.  Another  stated  that  the  report  should  be 
, omde  a part  of  the. employee's  permanent  personnel  folder. 


MOBILITY 


As  Indicated  previously,  the  Interns  spend  two  years  at  an  Initial  train- 
ing site  and  then  move  to  a permanent  duty  location  (POL)  for  the  third  year 
of  training  and  permanent  assignment.  Forty-two  point  zero  two  (42.02) 
percent  of  the  respondents  felt  that  this  was  a necessary  part  of  the  pro- 
gram. Often  made  comnents  on  mobility  Included  - 


"Mobility  gives  one  a broad  understanding  of  procurement."  (20  respond 
ents) 

"Each  MSC  Is  different  and  one  must  learn  new  ways  at  each  MSC." 

(10  respondents) 


( 


'The  mobility  requirement  Is  not  uniformly  enforced."  (8  respondents) 
'Early  notice  of  the  POL  Is  not  being  given."  (8  respondents) 


Ability  prepares  an  Intern  for  future  managerial  positions  or  shows  ^ 
• serious  comnitment  to  the  job."  (7  respondents)  - . u 

enthusiasm  for  training  interns  ! 

^0  will  be  leaving  in  two  years  was  detected.  However,  it  was  not  wide- 
require  eliminating  the  mobility  requirement.  Several 
express  a desire  to  reUin  good  interns  at  the  initial 
individuals  were  also  very  well  pleased  with 
the  tnird  year  and  graduate  Interns  with  whom  they  tod  worked. 


MHAT  DO  THE  INTERNS  THINK  OF  THE  PROGRAM? 


Of  those  answering  the  question  "How  well  does  the  intern  program 
prepare  me  for  journeywin  level  performance?".  18.58  percent  rated  the 
program  Mcellent."  The  majority,  57.38  percent,  rated  the  program  “good." 

seventy-seven  (20.77)  percent  rated  the  program  "fair"  and^ 
^ly  3.28  percent  rated  the  program  "poor."  The  longer  one  was  in  the 
intern  program,  the  better  he  rated  the  program.  None  of  the  graduate 
interns  rated  the  program  less  than  "good"  and  none  of  the  third  year  in- 
terns rated  the  program  less  than  “fair." 

Conroents  on  the  program  were  generally  complimentary.  Again,  some 
counts  wre  diametrically  opposed.  Four  individuals  indicated  that  more 
rotational  assignments  are  needed,  while  seven  individuals  stated  that 
there  should  be  fewer  rotational  assignments.  The  major  complaint  (28 
respondents)  ras  that  the  program  of  instruction  detailing  the  rotational 
WT  and  formal  sc.hools  training,  was  not  followed.  Related  to  this  question 
is  the  observation  of  20  respondents  that  all  levels  of  the  MSC's  need  to 
know  more  about  the  Procurement  Intern  Program.  This  is  especially  critical 
at  the  first  line  supervisor’s  level.  Other  frequently  made  conroents 
Included: 

^^^Jlnterns  need  more  work  than  they  are  currently  assigned."  (9  respond- 

. . . ' • 

"Interns  need  more  challenging  work  than  is  currently  assigned." 

(11  respondents) 

"The  Procurement  Intern  Coordinator’s  position  at  each  HSC  should  be 
staff«*d  full  time  by  a procurement  careerist."  (11  respondents) 

"Better  scheduling  of  rotational  assignments  or  formal  schools  train- 
ing is  required."  (19  respondents) 

Overall,  the  interns  are  pleased  with  the  program.  They  feel  that 
they  are  given  more  than  adequate  training  and  that  they  will  be  capable 
of  functioning  as  full-fledged  journeymen  at  the  end  of  the  intern  program. 


g 

0 


i:  ■ 


Ci 

1 

« 

g 

_ i 

u 

O 

(J 

O' 


ij 

I 

~ I 

iJ 

# 

_ I 
{ ) 


I, 

i 

1 

i 

J 

« 

« 

k 


o 

o 

o 

o 

p 

O' 

* 

o 

o 

• O' 

o 

0 

■(;_  ■ 
c 
(1 


WAT  DO  NON-IffTERNS  THINK  OF  THE  PROGRAM?  v' . f^ri  Asa  .5 

Several  Individuals  who  supervise  or  work  with  procurement  Interns  were 
Interviewed.  Better  than  90  percent  of  these  Individuals  believe  the  pro- 
gram does  an  excellent  job  of  preparing  the  interns  for  journeyman  level 
performance.  Only  2 percent  rated  the  program  as  "fair."  None  perceived 
the  program  as  "poor."  Many  of  those  Interviewed  remarked  that  each  year  ■ 
the  Incoming  Interns  seem  to  be  better  than  the  previous  year's  interns. 

' ‘ I ■ — - •*-..■«  --‘ii*  ../i.  X ^ 

Most  Individuals  stated  that  they  believe  the  Interns  are  given  Interest- 

• Ing  and  challenging  WT  assignments  considering  the  available  workload. 

Many  of  those  Interviewed  relied  very  heavily  on  the  interns  for  accomplish- 
ment of  their  office's  workload.  Nearly  all  felt  that  the  roUtlonal  OJT 
assignments  were  extremely  beneficial  to  the  intern's  career.  The  most 
connon  criticism  was  that  the  first  line  supervisor  was  often  uninformed 

of  upcoming  rotational  OJT  and  formal  schools  training.  , • ‘ 

• ^ 

The  majority  of  those  Interviewed  Indicated  that  the  formal  schools 
training  given  the  Interns  Is  more  than  adequate.  Several  stated, that  the 
ADP  Orientation  Seminar  Is  Inappropirate  In  that  the  Interns  need  an  ADP 
orientation  to  the  ALPHA  system  much  more  than  they  need  a general  ADP  i -sj 
orientation.  - 


CONCLUSIONS 


: > ■ 'ir 


i. 


- . r/u:  •02  - id 


1.  The  DARCOM  Procurement  Intern  Program  is  most  definitely  succeeding  In  „ 

producing  competent  jwrneymen.  Upon  completion  of  the  third  year  of  .• 

training,  the  intern  Is  capable  of  functioning,  on  his  own,  as  a qualified  ' 
procurement  professional.  • ^ ^ 

2.  OJT  rotational  assignments  are  of  the  right  type  and  are, allowing  ifer 

some  flexibility,  of  adequate  length.  . , - 

3.  The  formal  schools  training  Is,  with  the  exception  of  the  ADP  Orienta- 
tion Seminar,  excellent.  There  is  a problem  with  the  scheduling  of  some 
Interns  for  the  Defense  Small  Purchases  Course  and  the  OJT/classroom  con- 
nection Sn  the  Pricing  area  must  be  strengthened. 

4.  A valid  predictor  of  intern  Intake  requirements  Is  both  feasible  and 

sorely  needed.  . * * 

5.  First  line  supervisors'  are  not  always  aware  of  an  intern's  upcoming 
rotatlonpl  assignments  and  classroom  training. 

I 

6.  Increased  interaction  among  the  DARCOM  Procurement  Intern  Coordinators 

would  provide  a transfusion  of  useful  ideas  and  techniques  which  would 
continually  enhance  the  intern  program.  - ^ 

‘7.  The  Administration  of  the  procurement  Intern  program  requires  exten- 
sive effort  and  a knowledge  of  the  procurement  process.  < * - ‘ • 


( ) 
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th*t^  sufficient  for  ensuring 

fcJL".  '*  “ * !>•■:- 

!:<n!!!lJl!J  presented  for  alnost  every  «#eek  long  course  and  cor- 

fomal  recognition  1$  given  to  the  fact  that  ororurg^ 
■Ml  Interw  cavitte  . three  year  teteeslre  tratnlS  pr^S'  ‘ 

lu^fM  ^4”  *^^f**^  of  the  procurement  Intern  program  would  enable  HC  ■' 

."!Si:.e*?di:2r  »'  ^ Pr.9r-.“n';»"Sd  ■ 

>vryC- 

■■  -'•  ,.■:  ■'•-}■  y : ■»•,*!  •' 

•=r  ‘*  ,=  -.t  -io.  ( 

'•  ‘ 'V;  i.  _■  • 


b. 


d. 


the  mortality  rate  In  the  program 
WRCOH's  return  on  Investment  In  the  program 
mobility  patterns 

> * • i • • rr.  i 4 ^ . 

• r 

career  progression  • - 


RECOHHENDATIOWS 


■-  ■ ■ .‘r' . . 

■•  - - • '•:•-•■  • ■••■  -ViA-r-t 

r - 4 ' i 

1.  The  Automatic  Data  Processing  Orientation  Seminar  should  be  reolaced 
^■1  * course  lAlch  would  give  the  Interns  an  overview  of  ADP  but  with  stress 
^t^arh  MV  systai  and  the  uses  of  ADP  In  Procurement.  It  Is  felt 
iShous?  expertise  to  fconduct  this  type  of  course 

Directorate  of  Intern  Training,  ALMC, 
should  Initiate  an  enrollment  request  for  the  Defense  Small  Purrh^^p<  r^r 
respondence  Course  for  that  IntSn.  This  woSld  geHIJavS  ensuJe^t  ^ 
Intern  can  complete  the  course  piror  to  completion  of  Small  Purchases  WT. 

^ DARCOM  P«phlet  690-3-14.  "DARCOH  Civilian  Training  Program  for  Procure- 

^ to  require  that  the  Principles  of 

^tract  Pricing  course  be  completed  prior  lo  an  Intern's  OJT  In  the  Pricing 

continued°*"**"^  intern  requirements  forecasting  model  should  be 


!l  assig'wnent  and  classroom  training  schedules  must  be  fur- 

nl^d  both  the  Intern  and  his  supervisor  sufficiently  1.,  advance  to  allow 
planning,  pis  Is  essential  if  the  Interns  are  to  be  given 
aqy  type  of  meaningful  and  challenging  responsibilities.  * 

6.  A meeting  snould  be  held,  at  least  yearlv.  for  all  Prnnirfmrnf-  Tn4-a.>re 

^ DARCOM.ALMC  and  FPO  personnel  connected  with  the  pro- 

SJ^?ai  il  *^th  the  progrSr aSd^ 

potential  Improvements  In  the  program  would  be  discussed  • ^ 


irT 


’f- 


I ■ 


(■) 


ZI4  ^***  administration  of  the  procurement  intern  program  should  be  the  ' 
i^ponsibiiity  of  the  Procurenient  Intern  Coordinator  It  should  he 

Should  be,  .to  a large  degree,  based  on  Ms  efforts  In  this  area.^ 


8.  The  Career  Intern  Progress  Report,  DARCOH  Form  2002.  should  be  revised 


j-rt,  t 


1 CA*  *1^*^  • broader  range  in  the  rating  scale,  similar ^to  the  verA 

to  50  scale  used  on  the  Enployee  Career  Appraisal,  DO  Form  1017  (Test) 


perfS;maSS''^‘‘*  guidelines  or  criteria  uhich  mould  be  used  in  evaluating 


training^'*^'^  recording  of  grades  an  intern  earns  in  formal  schools 


J 


d.  make  the  report  a permanent  part  6f  the  employee's  personnel  folder 


’■■1. 


9.  A diploma  should  be  awarded  an  intern  upon  completion  of  the  program. 


tern  intake  the  procurement  intern  program  can  be  constantly  improved. 
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• '.  ii  ^ |?an- ^ . 

. -'\  f n ^»- ' i -.1  i?,'  • . \/  ^ «vrSOti^*r  .. 


First  of  i11»  we  mst  define  It*  Turning  to  Webster's  several  years  ago,  I was 


surprised  by  the  vagueness  of  ^thc  definition. 


ri.>\ 


A calling  In  which  one  professes  to  have  acquired  sooe 
special  knowledge  used  by  wqy  either  of  Instructing, 
guiding  or  advising  others  or  of  serving  then  In  some 
art  or  science;  as  the  profession  of  arm,  theology, 
law  or  aedlclne. 


♦:ijQ 


J 


’nil 

f 


>H?v- 

A CVVJ^r*  . 

rtr  - f -‘I 

This  seewed  woefully  Insufficient  for  conversion  to  an  objective.  Assimlng  that  there 
■ist  be  Measures  of  professlonallsw  or  criteria  that  nust  be  net  before  soweone  or  so«e 
group  annoints  your  field  as  a profession,  I turned  to  the  Federal  GovenwKnt  — It 
regulates  alwost  everything  and  was  sure  to  have  sonewhere  codified  what  It  takes  to  be 
a professional. 


The  Internal  Revenue  Service  Is  one  organization  that  serves  an  annointing  role;  If 
your  field  Is  not  recognized  as  a profession  you  cannot  deduct  education  expenses  for 
courses  that  you  take  to  Maintain  ^^gnofesslonal  currency.  Certainly  there  have  been 
occasions  where  such  deductions  h^t^  been  allowed  by  Individual  11^  auditors,  but  whenever 
the  deductions  have  been  disallowed  and  subsequently  appealed,  the  appeals  have  been 
denied.  A good  exanple  of  this  occurrence  is  the  ''profession  of  ams*  Included  In  Webster's 
definition  — the  IRS  does  not  recognize  the  "profession  of  arms"  as  a profession.  It 
recognizes  as  professionals  only  engineers,  lawyers  or  nedical  doctors  who  service  In 
the  Military; 'the  line  officer  in  the  Havy,  the  rated  officer  In  the  Air  Force,  and  the 
coobat  anas  ^flcer  In  the  Anqf  are  not  professionals  In  the  eyes  of  the  IRS.  How  then 
do  you  get  the  IRS  to  bless  your  field  as  a profession?  The  answer  received  was  that  the 
IRS  Gfe.iera1  Counsel  Must  rule  on  an  appeal  that  relates  to  professional  status  such  as 
a deduction  for  education  expenses.  Without  further  comment  on  that  situation,  let's 
accept  that  as  a required  wicket  through  which  we  must  pass. 


The  US  Civil  Service  Comnlsslon  was  the  next  source  counsulted  because  there  Is  a 
large  number  of  procurement  officials  In  the  Federal  Government  work  force,  and,  assuming 
that  they  desire  the  professional  status  conferred  on  their  counterparts  In  counsel  or 
accounting,  there  must  be  some  criteria.  The  CSC  (staff  personnel)  stated  that  a pro- 
fession Is  characterized  by: 


— 


A sophisticated  body  of  knowledge  that  can  only  be  possessed  by  the  professional 
and  Is  not  readily  available  In  the  form  of  written  regulation  or  procedure.  This  must 
be  reflected  In  CSC  position  classification  standards  and  personnel  qualification  standards. 


A work  force  that  Is  nde  up  of  Individuals  possessing  at  least  bachelor  degrees 
with  degree  concentration  In  the  professional  field.  However,  several  recent  court 
decisions  have  prompted  some  waffling  on  the  part  of  the  CSC  staff  In  this  area.  I asked 
If  personnel  management  was  considered  to  be  a profession,  and  after  sa»  pondering,  the 
CSC  staff  (personnel  management)  replied,  "No."  At  least  this  response  was  a bit  more 
direct  than  the  one  received  from  IRS. 
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The  first  criterii  set  forth  by  the  CSC.  "a  sophisticated  body  of  knowtedo# 
r^iw  fur^r  coammt.  Hcm  do  we  measure  this  requirement?  One  way  Is  to  look  at 
^1*  ‘**9fee  of  c^lexlty  of  our  field.  How  much  of  our  business  Is  perfonoed  bv>the-baok 
a regrlatlon  or  policy  docuaent  to  determini  how  tTSsjS  £ I 9^^ 

’ of^^^w  ■•***'^  exists?  Can  ir  find  the  answers  to  most 

these  docawents  or  do  the  anst#ers  cone  for  j donent  calls?  The  ansi  »rs 
^ • natter  of  Jud97<!nt  if  you  consider  the  amnt  of  contract 

. definition  requires  that  we  have  a work  force  wK,*  those  nersms  of 

SJtSt1n?f1^'’“Thiri?a‘^^«*  concStratlon  In  the 

field.  This  Is  a tougn  requirement  because  only  two  universities  offer 

P™9'‘ams  In  contracting,  and  many  years  will  pass  before  ; can  oenwrat* 
enough  demand  to  convince  the  universities  that  ^rlictSg  pKJraS  « nee5^3.® 

u c«T»^hot1oo  to  the  definition  of  professionalism  cows  frtm  U e>i<t*r<  «f  ^ 

AnoUier  sl^llne  of  Interest  to  WXJUL  should  be  the  accepted 

occupations  deal  with  customers  and 
professionals  with  clients.  If  there  a difference  oetween 
cutomers  and  clients?  If  so.  what  Implications  may  surt  4. 

^fference  have?  A customer  determines  what  goods  or  ^Ices 
!!**  V**  f'«e<1<M  to  shop  around  until  hi 

Pi^lse  that  he  has  the  capacity 
woJiSS*  if^Jhl!®^****"**  *?  Judgement  about  the  services  ^ 

<s  not  happy,  he  Is  free  to  criticize 
relatl!^!^^  fC**  *1?*  * refund.  In  the  profe:s1onal 

client  has  no  choice  but  to  accept  the  pro- 
client  has  surrendered  his 
^ professional.  In  this  way.  the  client  Invests 
Judgement.  One  cf  the  basic 
professionalism  Is  that  professionals  have  the 
tendency  to  ACQuIro  a Bionopoly  of  Jud^om^nt. 

Am  ^"Poctjnt  aspect  of  professionalism,  a matter  of  which  we  may  be  unaware 

Will  we^wlnf.  *11^  jud^ntal  authority  by  the  proliferation  of  pollcy^and  proSdure? 

2I»  personnel  administration  has 

orthrfff22"orh'®  **  mechanistic;  Ignores  the  omnlrresew 

advlm»rilr2i««^“^?  P’«“S  sUff  too  Often  In  an 


/ : 


! . 


Ut  also  Mist  consider,  froi  our  viet  of  other  professions,  the  need  for  discipline  to 
prevent  our  professional  status  froo  dioiiiishiiig  after  we  have  attained  it.  Consider  ti» 
profession  of  law.  liiile  driving  to  this  syoposiun,  I observed  a biaiper  sticker  on  an 
aiitoff>bi1e:  *0011*1  HIT  HE  Hf  LAWYER'S  11  JAIL.*  Another  indication  of  profesiional 


degradation  cane  frcn  the  Washington  Post.  In  his  review  of  one  of  the  current  books  on 


Watergate  (Stonewall  by  Richard  Bcn-Veniste  and  George  Franpton),  Robert  Sherrill  begins: 


- ^ 
niti 


Anericans,  in  the  privacy  of  their  bruised  hearts,  have  always  had 


r;  a hidden  allegiance  to  the  rite  of  lynching.  It  gets  the  Job  done  < * 

without  the  incunbnnce  of  lawyers. 


^ DUX  one  nust  oiscriKinave.  inem  are  young  lowyei  >,  wny  or  weiH, 

f Still  uncorrupted  by  their  profession,  look  upon  law  as  a systcn  foi 

^ speeding  Justice.  And  then  there  are  the  old  slick-haired  lawyers, 
" who  often  seen  to  forget  the  difference  between  law  and  politics 


or  law  and  Just  plain  wooRy-grubhing. 


Enough  about  definitions.  Where  do  we  stand  now,  idiat  progress  have  we  nade,  and 
what  oust  we  do  to  achieve  our  objective  of  professional  status? 


He  have  nade  progress*  The  National  Contract  Nanagenent  Association  sponsored  and  is 
operating  a certification  progran  for  all  contracts  wanagers.  The  progrwa  establishes 
a discipline;  it  holds  the  certified  person  to  a code  of  ethics;  it  neasures  attainwent 
of  experience,  education,  training,  and  knowledge.  The  education  requirewent  for  certi- 
fication is  a bachelor  oegree,  in  any  field  of  concentration.  In  the  future,  this 
requirewent  will  change  to  a bachelor  degree  with  concentration  in  the  field  of  con- 
tracting. We  cannot  at  this  tine  require  that  concentration  because  so  few  universities 
offer  contracting  prograns.  Incidentally,  the  requirewent  that  applicants  possess  a 
degree  is  probably  our  wost  inportant  step  toward  professional isa,  because  it  encourages 
the  universities  to  establish  degree  program  concentrating  in  contracting.  Given  a 
positive  response  by  acadewia  to  these  dewands,  we  nay  eventually  be  able  to  bring  into 
the  field  only  those  persons  with  a degree  specifically  in  contracting.  When  we  have 
achieved  that,  we  will  be  qualified  in  the  eyes  of  the  Internal  Revenue  Service  and  the 
Civil  Service  Connission. 

As  we  progress  tmard  professionalise  we  nust  keep  in  wind  the  future.  The  decisions 
we  wake  along  the  way  can  either  enhance  the  stature  of  our  discipline  or  prevent  us 
frow  being  able  to  reject  those  persons  who  violate  our  code  of  ethics. 

There  is  o^e  last  concern:  the  need  to  naintain  the  Judgment  factor  in  procurement  and 
contracting.  If  your  research  culminates  in  wore  definitive  policy  and  wore  procedure  that 
reduces  decision  waking  or  if  you  encourage  a mechanistic  approach  to  contracting,  you 
will  be  hindering  progress  toward  professionalism.  On  the  other  hand,  if  your  research 
endeavors  promote  decision  waking  causing  in  all  instances  a judgiwnt  call  by  the  procure- 
went  official,  we  way  someday  be  calling  him  a procurement  professional* 
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}c:ut 

>:  * . ;S;^5  ? ?if>/Sett-* 


‘ **^^'*  **  ■ 5 ■'•'c-iw  ^0  ncfr*  Tvuii^ 

bMlsK  'ti  If 


•V  general  Uiene  tkis  nonilng  refers  to  need  for. new  talent  In  procuromt  profession, 
••ords.  about  ow^hlrd^  tte  ^ “*««*«»•.  WW.  In  other 

.f  .M.  „„  m 

«■»  ^ift  „»«(» 

•Viors  but  EnolneersToperatlon  Res«rcht«  Business 

quantitative  backgrounds.  I don’t  Les'fotti^nM^i^H * 1"^  ***'**c*'  en^ 

type  of  talent  to  which  i rete^  ‘ ^ettlg  personally  but  I can  say  that  he  Is  the 


Ifc.’t  ire  they? 


*•  A ^ • 


I.  ~t  I...  hi«  „ o»  T*. 

-«gory  in  tne  past,  although  a great  deal  of  progress  Is  currently  being  nade. 

..i.«-'rr;:^c';rSnr'Hri2r«  r”  '»*  '■“ »» “»■’• 

itout?  Damn  few.  ^ ^ would  even  know  what  you  are  talking 


aware 


I Ito’t  bellm  It  II  tecs,  a*,  !««>.««;  tPt.t  sinply  tt.,  Mt 
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Director)  up  to  about  1970  seeaed  to  place  the  procurewrt  area  scaeuhat  below  the  other 
functional  areas  of  ensineering.  finance,  etc.  However,  with  the  advent  of  Fixed  Fee 
Contracts  and  incentive  contracts  the  procurenent  function  has  rapidly  gained  recognition 
as  a solid  career;  at  least  with  the  insiders  but  less  so  with  the  young  ulent  Just 
entering  the  labor  warket.  c ,»i-  . . 

I'll  address  only  one  subset  of  the  foraer  job  attributes  (educational  opportwities) 
and  leave  the  second  attHbute  (opportunity  for  advancement  prowotion.  etc)  to  others. 

I'll  also  Halt  ay  reaarics  to  only  one  aspect  of  echication  - Graduate  Frocureaent 
Education.  ‘ ^ 

I ^ . f .'.  • r ^ 

Or.  Hood  has  addressed  aspects  of  Job*re1ated  learning  projects  and  Ed  Lovett  has 
discussed  the  DARCQN  Procurenent  Intern  Training  Prograa.  Graduate  level  procurenent 
training  conplenents  these  prograns.  ' ■ 

. r''  • » ^ ^ •*’1^*^  S'*:** 

Let  m briefly  describe  the  evolution  of  such  a graduate  degree  prograa. 

In  1967  the  Cape  Kennedy  area  recognized  a need  for  procureaent  training  and  there 
aas  virtually  none  aval**Me.  This  resulted  In  the  FIT,  In  cooperation  with  the  XCNA, 
establishing  a certlflca.^  prograa  In  the  procureaent  area.  However,  several  Individuals 
In  the  procureaent  profession  expressed  Interest  In  advanced  work.  So.  In  1968  FIT. 
again  In  cooperation  with  the  NCMA,  lapleaented  a Raster  of  Science  degree  prograa  In 
Contract  and  Procureaent  Ibnageaent. 

At  first  all  of  the  courses  were  In  Contract/Procureaent  Ranageaent.  This  didn't 
aake  auch  sense;  there  was  too  auch  redundancy,  too  auch  overlap.  He  aust  recognize 
that  first  we  aust  be  a professional,  effective  oanager  and  then  a functional  specialist, 
a procureaent  professional.  With  this  In  aind  the  curriculum  was  changed  to  include 
traditional  Business/Nanageaent  courses: 

Cconoalcs  . . 

Finance 
Accounting 
Behavioral  Science 
MIS 

Organization  Development 

» 'Ik*-’. 

Plus  strong  concentration  In  Contract  & Procurement  Management. 

Most  of  these  are  probably  familiar  to  you. 

Basic  Contract  Manageftent  and  Administration 
Contract  Changes.  Terminations  and  Disputes 
Cost  Principals 

Contract  and  Sub-^:on tract  Foma t1  on 
Contract  Negotiation  and  Incentive  Contracts 
Research  Seminar 
Contract  Management 

Plus  three  electives  in  Contract  Management  or  Business.  ^ ^ 

We  tend  to  modify  the  curriculum  as  needs  change  but  the  mix  of  contract  management/ 
business  courses  stays  pretty  much  the  samel 
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Incidentally,  we've  had  good  results  la  offering  an  Undergraduate  course  In  Procurewnt 

i:>  • , .f  I,-., 


V'itjion  or  less  covered  the  %e11>defined”  aspect  of  the  Educational  need  since 
progranT*"***  •‘xwt  ^ years  to  a substantial  ly  highly  structured 

^ ^ .t  m.  it 


•"<*  "®-  Gr^Awte  offerings  In  ^ procure^t 
^Id  are  not  ciwonplace.  One  sethod  of  delivery  which  1 feel  that  FIT  has  had  a oreat 
success  In  Is  In  the  Inplenentatlon  of  off-cavus  prograas  as  coeoerative  effarrc 
nith  such  l^tltutlons  as  Amy  Logistics  "^ge-ent^Sr^TSSTtenSeSySSjLriS 
Training  Agemy.  and  the  TransporUtlon  Sctool.  t^neenng 


These  iMve  Bich  to  offer  - inllLa  our  science  and  engineering  proorMS.  then  Is  no 

a^^esttotlml^^lln^  1^«tories.  The  offerings  occur  la  centers  of  high  Interest 
and  exceptional  talent.  He  have  a nuaber  of  real  advantages  in  theM  offerings: 


. -12  readily  accessible  to  a larger  body  of  students  who 

afford  the  year  required  to  attend  at ■«» 


*)  Availability  of  exceptional  talent  (Practitioners)  for  Instruction  In  the 

£'s.frrrs.*«rsr7‘ " •* 


s . sr 


^ 4)  A blend  or  alxture  of  both  students  and  faculty  froa  all  sides  of  the  house: 


Industry 
Government 
rivil  Service 
MlllUry 
NASA 

Universities 


a fai  program  benefits  by  such  contact  and  ends  up  with 

I S^-td^Sar?  ^SolpSr^  *"  ™ Pn>cur«ent  process  thru  the  day-to-day  contact  In 


.**  *"  * see  two  attributes  to  be  net  In  the  Procuravnt  field  In  order  to 

“ •- 


Tw  described  the  evolution  of  one  aspect  of  the  educational  prooran  - oraduate  lewi»i 
r successful  approach  to  «ke  tS?^  edu«[?Sn 
available  and  1 ve  noted  a few  of  the  nore  evident  advantages  of  such  a progrM. 


rw eppro*®***  to  this  problen  I'm  sure,  but  ny  experience  at  this 
"T  opinion  that  graduate  level  procureaent  training  Is  greatly  In  demand 
•nd  can  be  delivered  effectively  by  our  university  system.  ® ^ 
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CqCLUDING  REMARKS 


John  K.  ICtfisealller. 

Oialraan,  UOD  Procuromt 
Research  Coordinating  CoMrittee 


000  Is  big  business  - we  need  research.  There  Is  nothing  so  useless  as  doing  things 
idth  great  energy  and  efficiency  which  should  not  be  done  at  all.  That  Is  probably  the 
greatest  pitfall  we  wst  avoid,  both  In  wanagenent  and  1n  the  research  business.  Does 
this  then  wean  that  every  research  project  wust  have  an  lawedlate  and  total  payoff? 

Quite  obviously  the  answer  Is  a resounding  no.  A research  project  which  Identifies  a 
"dead-end*  has  value.  To  keep  ahead  of  the  proverbial  power  curve  we  nust  have  a mix 
of  projects  that  are  of  the  problem  solving  type  and  from  idilch  we  can  expect  an  immediate 
payoff  as  well  as  long  range  efforts  which  are  primarily  design  as  thought-producing 
In  today's  environment.  The  acquisition  process,  and  that  Is  what  I think  procurement 
research  Is  all  about.  Is  an  extremely  dynamic  process  that  is  continually  affected  by 
Congressional  action,  executive  directives*  and  the  economy  as  well  as  the  socio-economic 
and  geo-political  sltiatlon  on  a world-wide  basis.  Ve  have  but  to  look  at  the  F-16 
program  with  Its  International  implfcatlons  to  realize  what  effect  Geo-Pclltlcs  can  have 
on  the  acquisition  process.  Thus*  our  program  must  be  a mixture  of  the  applied  as  well 
as  theoretical  research.  However,  It  Is  well  for  us  to  be  constantly  aware  that  our 
first  priority  Is  to  be  responsive  to  the  requests  of  our  customers.  You  have  been 
challenged  with  new  thoughts  or  new  slants  on  old  Ideas  - research  must  respond. 

As  Chairman  of  tne  Procurement  Research  Coordinating  Coomlttee  and  through  wy  overall 
Involvement  In  the  procurement  research  program,  I have  had  the  pleasure  of  meeting  most 
of  our  full  time  research  staffers  and  mny  these  past  few  days.  From  my  observations 
I have  concluded  that  we  have  an  extremely  talented  group  who  not  only  understarui,  but 
of  even  greater  significance  have  a strong  feeling  for  the  acquisition  process.  However, 
the  success  of  our  program  needs  more  than  able  people  ably  led  If  It  is  to  be  effective. 
The  one  additional  Ingredient  I'm  talking  about  Is  the  method  you  choose  to  present  your 
results.  Your  clients  will  be  functional  managers  and  these  Individuals  are,  for  the 
most  part,  not  research  specialists.  Consequently,  the  product  they  rightly  expect  Is 
one  that  tells  them  In  laymen's  terms  what  your  recommendations  are,  how  and  why  you 
arrived  at  your  conclusions,  and  how  your  reconmendatlons  derive  therefrom.  I'm  sure 
you  operating  managers  will  agree  with  me  that  there  Is  nothing  more  bori.ng  than  opening 
a report  that  Is  loaded  with  mathematical  formulas,  theorems,  hypotheses,  etc.,  that  have 
meaning  only  to  someone  else  in  a specific  field  of  research.  Skepticism  Is  the  natural 
result  and  a communications  barrier  Is  built  up.  He  probably  won't  look  beyond  the 
technical  data.  In  making  this  statement  I am  fully  aware  that  operations  research  tools 
form  the  basis  for  many  of  your  conclusions.  This  Is  good  but  what  I am  really  asking 
you  to  do  Is  to  place  yourselves  In  the  functional  manager's  position  when  the  report 
Is  prepared.  Sectlonallze  the  report.  If  you  will,  but  give  him  a clear  story  that  can 
be  understood  In  his  terminology. 


7i; 


-'mr 


no  *5*  always  draws  attention.  Then  it  is 

^ research  has  centered  on  ell  phases  of  the 

S?  ?!  *^***?**  ^**  * " -•***  ^ gla»orous  pr«ur«ents. 

th^^si.!M!  mJ2  us  to  reweeher  that  the  cost  of  supporting  the  weapon  systen 

s.TjS'.Ji.j's.s  •J.i.t'?" 

"• “ »«  SsSJcSW.'.trs 

ini  ?***  support  services  that  mist  be  perforwd  to  keep  the 

^ repair  contracts  area  there  are  problen  with  types  of  con- 

S terirn  S5?*alrf'th^  «rt***?ii*S  ®'‘*”’  ‘®  ^ ®'  SPUCJ^^WS  «"<» 

jucn  urvs  as  ove.  and  above  work.  The  base  support  services  are  hfahiv  c«ihtit{i/o  mh 
oreate$t*^J[t2t<M^^r****  probl*  of  ‘buying  dn."  Also,  in  this  area  we  have  our 

fracts  awS3S"«de*Xu«  1(.To*mS  Sll'JulKs^fct"  TqlStiSHTSiIld'likrt'o 

K)  Jf«S5:cSr*iS*re*v"fSl‘ai5  J^iSS  . 

A specifications  for  total  contractor  operation  of  a base  can  ^ hrt^^k 

SjoJfSnuf  Jo'b“?d1'"S 

opportunity  to  bid?  Or  nust  we  go  the  single  total  package  rout**  and  lustifv  tiii« 

“*  .Sr Z*-. 

hMdl  “**  hereuse  of  our  stickiest  problems  in  procureont  .ear  their 

heads  in  the  weapon  and  base  maintenance  and  support  service  contracts. 

■ 1 

Ladles  and  Gentleaen,  all  I m trying  to  say  to  you  is  look  over  the  Mitire 

r“;r^  •”  “*  si^ 

ine  rewards  and  payoffs  fro«  a Job  well  done  are  there.  ^ 

mJS  r'n.si'.TrJr 

jrj;g  fir  iL“:5  s,r:  sss;"’"’ ' 

r7>«arcn  is  too  i^>ortant  to  be  left  only  to  researchers.* 

Mow  can  we  tell  the  world,  or  just  tell  the  boss  how  good  we  are?  Our  OSO  Directive 
wns  for  an  annual  summary  of  the  procurement  research  effort  with  quarterly  updates.  The 
Defense  Logistics  Studies  Information  Exchange  (OLSIE)  Is  the  source  since  each  research 
proj^t  is  to  be  Initially  reported  and  updated.  Therefore,  suimarles  are  already  Included 
In  their  ccaputenzed  system.  But  be  sure  that  each  of  you  logs  In  with  DLSIE. 
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I ) ■■■  ■ •■  ■ * .4., 

' In  aiQT  case,  we  need  a vehicle  to  *te11  and  sell*  top  Mnapeoent  on  what  is  90109 
on.  Unfortuutely,  I don't  hai«  the  co^>;ete  answer.  In  fact,  it's  up  to  each  of  you  to 
*te11  and  sell.*  You  can  bet  your  bottos  dollar  that  this  is  an  itca  we  in  OOO  will  push 
and  push  hard  at  future  PRCC  neetings  until  we  do  cow  up  with  an  answer.  1 consider  this 
crucial  to  the  continued  health  of  the  000  Procureaient  Research  Prograa.  As  the  FPl  ■ 
becows'a  fact,  we  can  look  for  an  assist  in  this  area.  Other  people  are  helping  too. 

' Keep  yourself  inforwd  • tap  into  the  results. 
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At  Thursday's  luncheon,  the  National  Contract  Hanagewnt  Association,  which  has 
done  auch  to  promote  developaent  of  a body  of  procurewnt  literature,  made  their  annual 
awards  for  the  top  procureaent  research  projects,  lie  thank  NCNA  for  their  interest  and 
assistance. 


I want  to  recognize  Dr.  Paul  Arvis  and  NAJ  Place  for  the  outstanding  organization 
and  support  of  this  conference. 


The  7th  Annual  Syiposiua  wi .1  be  hosted  by  the  Air  Force  - Nay/June, 
place  to  be  announced.  There  ^11  be  a call  for  papers  in  Nov/Dec. 


Tiw  Md  * 


I thank  you  for  attending  and  l.ope  yon  enjpyed  the  progr< 
Have  a safe  trip  how. 


M.  . ■ ■ ♦ V'  •■■■ 

« * «••■.  ■■  i.. 

■ . V I.  *'  --/•  ' 
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NCMA  AUAROS  FOR  BEST  PAPERS 


m 


'>r  -vTo.  >/» 


This  year's  NCMA  awards  for  best  papers  went  to  the  following  individuals: 


^ * rt* 


Chester  Rees,  US  Army  (Enroute  to  Germany) 
An  Analysis  of  the  Revised  Rate  Structure  for  Civilian  EnDloyee  Benefits 
included  in  0MB  A-76  Cost  Comparisons."  (Page  715).  v 


^ 4 


2d  Prize  - $200  - Major  Felix  M.  Fabian,  Assistant  Professor,  Department 
of  Economics.  Geography  and  Management,  US  Air  Force  Acadany.  "Mixed 
Procurement  and  the  Management  of  Risk."  (Page  608). 


^ Prize  (Tie)  - $50  - O.M.  Satqrer,  Supervisory  Electronics  Engineer, 
Naval  Electronics  Systems  Engineering  Center.  "The  Procurement  and  Manage* 
ment  of  a Government  Owned/Contractor  Operated  (GOCO)  Electronic  Repair 
Facility."  (Page  312).  ^ 


UA  " •‘osfiwy  E.  Howard,  Equal  Employment  Specialist, 

HQ  Defense  Logistics  Agency,  Executive  Directorate  Contractor  Employment 
Compliance.  "The  Application  of  »bnagement  by  Objectives  in  the  Contractor 
Employment  Compliance  Program."  (Page  278). 


The  presentations  were  made  at  the  luncheon  on  23  June  1977  where  Mr.  Jack 
0 Leary,  Executive  Director,  NCMA,  personally  gave  out  the  awards. 
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M amlysis  of  the  revised  rate  structure 

PWCTTIUAOjgLO^^  IH  M a-tk 

flBT  CflHPAaiSOkS  


Captain  Chester  L.  Rees.  Jr.  ' 
Uift€d  States  Ansr 
(£a  route  to  Korea) 
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THE  PROBIEH 


■S  ^ = 

It  Is  federal  procurement  policy  to  rely  on  the  private  sector  fbr  nrrA-ji  mnHc 

contract  out  versus  retention  In^iouse.  This  poficy  las  1i»le*nte?l2^ 
19M  and  subsequently  revised  In  1967  by  Office  of  Hanaoewnt  ^ cTmT^^l  a tk" 

««  »».t  .XSIiTrJj^  !&.X“?JS*cS*:s.h2»*!2.?SS»2;  • 

analyses  of  the  ONB  A-76  pragrw.  «^nsons  nave  provoked  recent 


”^].r 


^9«st  30,  1967,  para.  5. 

MDid. , para  7. 
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CimiAlf  BQttTIT  RA1ES 


ItetlrewBt  ( DlMbility 
HMltti  Insurance 


> Life  Ihsvrance 


nmssi  OF  nTTESTiwiow 


‘ - *1  j »•-  b'< .>  w*  .•  •?:  'ie  i*  47^., NK 


( Sheet. 


The  purpose  of  this  Invcstigatlofi  Is  to  deterofne  the  extent  to  ehldi  recent  cost  * 
OMpeHsons  thil  be  altered  by  application  of  the  revised  civilian  ciployee  benefit  rates. 

probable  effect  of  the  revised  rates  oe  futore  cost  coitoArlsons  and  the  resulting  - 
In-house  or  contract  out  decisions  will  also  be  detersfned. 

: r - - . 

NTPOTHESIS  ••  • ••  ' • : • • 

Tht  (vpothesls  to  be  tested  1$  that  the  education  of  the  revised,  eore  realistic 
casts  of  civilian  govenaent  e^loyees*  fringe  benefits  to  previous  U.S.  Ar^  cost 
co^>ar1sons  will  Indicate  that  a sl^ilflcant  niaber  of  decisions  would  have  ben  reversed 
In  favor  of  contracting  out. 

siaiinagicE  of  research 

Cost  conarlsons  are  used  to  base  decisions  for  In-house  perfornance  froa  one  to 
three  years;  at  the  end  of  which  tine  a renewal  cost  oo^tarlson  oust  be  conducted  to 
Justify  further  In-house  retention.  Since  there  are  relatively  few  new  In-house  starts 
each  year  as  opposed  to  renewals  of  existing  pragraas.  the  preponderance  of  future  cost 
aaeMrisons  will  be  conducted  on  prograas  already  tn  existence  which  are  the  target  of  this 
study.  This  research  project  should  provide  a bels  for  future  personnel  planning  based 
on  the  nuiher  of  expected  rewrsals  of  current  In-house  prograas  In  famr  of  contractlog 
out. 

Also  this  study  should  enable  agency  planners  to  specifically  Identify  Individual 
cost  co^Miisons  tdilch  aay  becoae  aore  costly  on  the  In-house  side  than  the  contract  side. 
If  a s1^>le  decislonrule  could  be  established  for  Identifying  which  CITA  functions  would 
pr^ably  be  affected,  another  problea  could  be  solved.  The  problea  has  derived  froa  a 
•Ally  27,  1976  OtB  aeaorandua  which  orders  all  federal  agencies  to  Idwitlfy  at  least  five 
fifictlons  presently  perforaed  In-house  to  be  contracted  out  to  private  Industry.^  Kith 
the  availability  of  a reliable  decision  rule  for  Identifying  potential  In-house  reversals. 


^U.S.  Af^  Training  and  Doctrine  Coaund.  TRAOOC  Paa>h1et  235-5  Cost  Ataivsis  Mort- 

rteet.  Fort  ftonroe.  VA:  Oaober,  1573,  pp.  4-5.  

..“Assistant  Secretary  of  Defense  (lU.)  Heaorandue.  Subject:  'Revised  Rate  Structure 
for  POD  Instruction  4100.33.  August  27,  1976,  p.l.  

•"fiord  Orders  (tore  Contracting-Out*.  Federal  Tiaes.  August  23.  1976,  12:  I,  16.p.l. 
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tht  «9mc1es  could  satisfy  Oit  requIrcMnt  tritiiout  9Dln9  ttirouah  tlie  tfue  coasubIm 
process  of  applyinp  ^ to  eadi  of  thalr  currmt  cost  ooi^aHsons. 


irmocoLOGr 


Test  data  for  this  stu((y  has  been  acquired  frcw  a sai^le  of  ^3  cost  co^rarlsoos  froa 
MRCOfI  (28)  and  TMflOC  (15)  Conerdcl-IndustHal  Type  Activity  (OTA)  Offices.  The  sa^ile 
was  taken  froa  cost  ooqpaHsons  froa  FT  74.  75.  and  76.  All  cost  caapaHsons  used  to 
saiple  had  been  audited  by  the  Anv  Audit  Agency  and  approved  by  the  cooaaid's 
“ Wosed  to  BMiy  whldi  haJ  been  subaitted  without  approval  to  date.  This 
^uld  Insure  that  the  saa^e  will  Include  the  aost  accurate  daU  on  idilch  to  base  the 
nndfop  of  this  report.  The  saqile  cost  am>arisons  will  be  analyzed  using  the  new  28. 7X 
benefit  rate  as  opposed  to  the  old  8.44X  to  deteralne  If  a substantial  nuaber  of  decisions 
<Mld  have  ^ reversed  In  favor  of  contracting  out.  A "t"  Test  for  significance  will 
then  be  used  to  deteralne  the  sutlstical  significance  of  the  war  value  for  In-hMise 
cmts  tefore  and  after  the  revision.  If  the  sa^ile  does  Indicate  that  there  will  be  a 
sIgMfIcant  m^r  of  reversals  for  future  cost  coapartson  decisions,  this  study  will  also  ’ 
eaaalne  the  .validity  of  the  revised  rates.  . ^ . 


ft  Is  e^cted  that  this  studywill  show  that  there  will  be  an  Increase  In  future  con- 
tractlng  out  decisions  by  the  Anqr  based  on  analysis  of  past  cost  co^iarlsons.  This  will 
nean  a decrease  In  the  nuPber  of  1n>house  prograas.  and.  consequently,  a decrease  la 
govemaent  eaplqyed  civilian  ampower.  • 


AhALTSIS 

SAlffllhS  TEOillQUE 


--r/*  1^  2 -a: 


^7hi$  stu<^y  Is  based  on  data  originally  collected  as  part  of  a research  project 
conducted  as  a requireaent  for  a Master  Science  degree  pmograa. 
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^1e  ^fch  lud  a nore  even  <HstHbut1on  over  the  past  three  veers.  This  was  ewialned 
hy  the  fact  that  IMRCON  had  Increased  Its  surveillance  of  the  cost  conparisons.  lequirfno 
■ost  to  be  renewed  during  the  past  fiscal  year.8  ' . . % . ® 

A second  Inconsistency  can  be  found  In  several  Instances  tdiere  the  costs  listed  on 
*®**  c®*t  conparlson  wrfcshMts  reaaln  the  saw  for  four  years  uhlle  others  Increase  froe 
year  to  year.  An  exai^le  of  this  difference  can  be  seen  by  examining  DARCQN  sawles 
m^r  6 (Redstaw  Anenal  T 801)  and  nunber  1 (Savanna  Arw  Depot  S 724).  The  explanation 
for  this  occurrewe  Is  that  nhlle  AR  235-5  does  provide  for  escalation  of  costs  year  to 
year  based  on  Inflationary  trends.  It  1$  not  required  In  order  for  the  cost  coapaHson  to 
be  approved.9  • . 

- Inconsistency  can  be  fbind  In  DARCOH  sanple  nuiirer  27  (Savmau  Arniy  Depot 

T 812)  rdiere  the  decision  was  aade  to  retain  the  function  In-house  even  though  total 
In-house  costs  are  nore  than  contracting-out  costs.  It  Is  evident,  however,  that  first 
yew  sUrt  iq)  cosU  for  the  govemwnt  were  the  priaary  cause,  and  If  the  walysls  were 
carried  over  to  a fifth  year,  the  govemnent's  savings  of  over  560.000  per  year  would 
reverse  the  touls  In  favor  of  retaining  In  house.10 

ftice  the  dau  had  been  collected  the  next  step  Involved  adjusting  the  c1v111«i 
personnel  costs  and  the  total  goverewnt  costs  by  the  revised  frtnge  benefits  rates.  As 

the  first  year  of  the  cost  cooparison  wortsheet  at  Exhibit  A Is 

tabulated  In  Table  2. 

TABLE  2 ■“ 

CCHPUTING  m CWILIW1  PERSOTOCL  COSTS  . ' T 


Civilian  Personnel  Cost 
Kcw  Civilian  Personnel  Cost 
Govemoent  Cost 
New  Government  Cost 


448^421  f 1,0844  = 413,520^ 
413,520  X 1.287  « 532,200 
1,875,162  - 448,421  » 1,426,741 
1,426,741  . • ,200  = 1,958.941 


PRELIMINARY  RESULTS 


Ihice  the  new  fringe  benefit  rates  had  been  added  Into  all  of  the  saaplo  cost  conpar^  — 
2;!  ^ ^ samples  (Ft.  Bermlng 

1*  ^**1^  ^ DARCOM  samples  (Sacramento  Aray  Depot  S 724, 

T ^ ^£I®I  Picatinny  Arsenal  X 931,  Tobyhanna  Army  Depot 

T 801,  Fort  MonaDuth  S 714)  had  reversed  Indicating  that  contracting-out  Mould  be  less 

Savanna  Army  Depot  T 812,  remained  more  costly  In-house;  however, 
5“"*^  Incurred  durtng  the  first  year  are  already  spent. 

^ contintje  the  function  In-house  due  to  the  $53,000  per  year 
savings.  Following  the  same  reasoning,  two  other  functions  which  had  toUl  contractor  costs 

CITA  toortinator.  Installations  and  Services  Directorate,  Head- 
quarters,  DARCON,  Washington  DC,  Interviews  October  1976. 

5lb1d. 

lOlbld.  • * 
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substantially  higher  93«erTU«t  costs  duHng 

iTu^Jefore  ?!I*  *•;«"«“  Sill  J 512.  Mcatlnry  Arsenal  T 801). 

. '”  ““  <tec1s.«s  »ild  b. 


• * * \ 

Ml*  saa^led  aTA  functions  that  should  revert  to 

the  private  sector  for  performance  under  the  revised  benefit  rate  structure. 

PROPORTION  TESTSlI 


:z. 


fZ  t' 


•-Cl  T?  We-HjV 

4.-^  - A Y- * i f n 


saaole  md*^  *®  ?*?  ****  the  TRAOOC 

nSiS  iri  Hi*  s“Ple  to  determine  If  their  relative  change  varies  sloilflcMtly. 

5S  decision  reversals,  or  20*  of  the  15  t^l  sailed  MsesTiid 

OAR^  had  7 decision  reversals,  or  23*  of  the  28  total  sanpled  cases.  Tie  difference 
naiJ^  £?I«kf**iL®^  small,  and  a sH^tly  higher  percentage  of  reversals  at 

*^****?  ^ B^Jonty  of  the  sallied  cases  are  from* 

2te'^~  auditing  proredures. 


— OM  %aic  m ■ icrCTlvC  DC 

^'wrsals  for  the  tuo  sauries,  a proportion  test  can  be  used. 


«o 


.fe  .\). 


t-'-V 


.05 

35 


.08 


\ 




"d  "t 
Do  not  reject  1^ 


2-  21  . 3 

55 . 55  *15 

“53 


12 

J? 


IT 


I.-  ?*  ^ government  costs  of  the  two  sarnies  can  also 

Ss^f^^^SWlfS^toS  any  sutlstical  difference.  The  mooc  mean  cost 

Tvi^nAo?!?..*' ***^’’^  ^ $1,906,375  or  11*  increase  vrlth  the  application  of  the  new  rates 

2^t‘?S2”f?:2"$r783T79  to 

1 ^ *ru»  lo  >^oJbo»ooi  or  15X.  A two  nean  proportion  test  can  alsn 

Md^M  ‘**^™*"*  "7  statistical  significance  In  the  two  Increases  of  11* 


f ) 


Ho 


«r 


(.lb  - .11)  - 0 


.05 

"7106  = 


W : 
C) ; 


.47 


*D®D  * 


n_ 


N "H 

Do  not  reject 


(.16)  (.84)  ♦ (.m  f.W) 


28 


o . 

I 

_ I 

u.: 


Allen  Sanderson,  College  of  Ullllam  and  Mary,  Econoarics  edpartaent  Is  recognized  — 
for  his  assistance  In  making  statistical  analyses  used  to  support  the  findings. 
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. . to  th#  b«s1s  of  thes*  tw  proportion  tests  It  is  therefore  concluded  that  the 
too  Independent  swles  at  CMUXM  and  TRAOOC  are  not  sUtistically  sioifficMt  in  their 
differences,  and  further  analysis  will  include  daU  fron  both  saiBles  coto>ined  into 
one  staple  of  43  cost  co^tarisons. 

*!^^*.“****  ***"•  • reversed  in-house  decision. 

K fmt  o^rvation,  10  out  of  43,  or  23K,  appears  to  be  a significnt  proportion,  aid 
s S*  proportion  test  will  be  used.  The  null  hypothesis 

in  this  test  will  be  that  10  reversals  is  no  nore  si^iificant  than  aera  reversals  or: 


o o 


4 * 

r 


rTi.  ^ : 


» 0 
9^0 


>r  ^ 


F-  Pen 


T5 


5.61 


10  35 


43 


• • teject  and  accept  ^ - irlth 
^ is  significant* 


than  99%  assurance  that 


Based  on  this  saaple  there  1$  sUtlstIcal  significance  In  the  ntnher  of  la-house 
tactions  that  Mould  have  becooe  aore  costly  to  the  govenMent  to  retain  In-house  than  If 
the  funct^M  mis  per^raed  by  the  private  sectDr»  and  thus,  the  hypothesis  of  this  study 
■s  proved  true* 

PAIR  DIFFEIOICE  SlflUnoWCE  TESTS 


Wiile  this  last  proportion  test  does  prove  that  10  out  of  43  is  a si9i1fic»t  nun- 
Z**  nuch  statistical  basis  to  predict  any  nuriwr  of  reversals 

« 4.5“'  ®4  coiparlng  govemnent  costs  before  and  after  application 

of  the  nm  tomfit  rates  through  neans  of  pair  difference  tests,  sone  prediction  can  be 
■ade  as  to  tdiich  cost  conparisons  will  reflect  higher  In-house  costs  than  contractor 
costs* 


xdttff 


—1  “i 


TT 


23M3  s S41.93 


Si  - 5>' 

n^l 


57,120.171 
45 


1166*2 


Taking  the  data  compiled  In  Table  3,  the  significance  of  the  difference  In  eeans 
and  after  the  revised  benefit  rates  were  used  can  be  analyzed.  The  noil  hypothesis 
In  this  case  will  propose  that  there  is  no  sigfilficance  to  the  difference  in  govemaent 
costs  before  and  after  the  revised  benefit  rates  have  been  applied* 
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U66.02 
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■ .*.  Reject  ^ and  accept  %rtth  •ore  than  99X  assurance  that 

r* 

there  is  a significant  difference  in  nean  govenwent  costs  hefore 


and  after  the  revised  rates  are  applied. 


- ■5* 


:t  • jF.f‘5.1’ 


}•. 


Another  pair  difference  test  can  be  used  to  determine  the  average  percentage 
Increase  In  1n>house  costs  and  the  significance  this  Increase  aay  have. 


j 


• From  Table  4 the  standard  deviation  fw  the  percentage  rise  In  govemnent  cost 
can  be  coaputed  as: 

- r -34  •r 

e JV  VJ. 
'’sf  ar  “ 


f 


589.83 


3.75 


42 


As  previously  noted,  this  study  has  gone  to  great  length  to  Insure  that  the  sanple 
drawn  was  random.  Assuming  that  It  Is  a random  sample  and  that  the  oean  and  standard 
deviation  of  the  percentage  Increase  In  government  costs  approximate  those  of  the 
population  of  U.  S.  Any  cost  coopaiisons,  then  the  following  predictions  can  be  made 
based  on  Table  5: 

1.  If  government  costs  are  within  4.7X  of  contractor  costs  before  the  revised 
rates  are  applied  then  there  Is  a 99X  probability  that  goveroBent  costs  will  exceed 
contractor  costs  after  the  new  rates  are  used.  Likewise,  there  Is  a 95X  probability  of 
reversal  If  the  Initial  costs  are  within  7.3S:  a 90X  probability  of  reversal  if  the 
Initial  costs  are  within  8.6X;  and  an  84X  probability  of  reversal  If  the  Initial  costs 
are  vrithin  9.7X. 

2.  Conversely,  If  Initial  government  costs  are  more  than  20.26X  (the  maximum 
possible  Increase  If  all  government  costs  are  for  civilian  personnel)  above  contractor 
co^-ts,  then  there  Is  a lOOX  probability  that  there  would  be  no  reversal.  Also,  there  Is 
a 95X  probability  that  there  would  be  no  reversal  of  costs  If  Initial  cost  differences 
are  nore  than  19. 6X;  a 90X  probability  that  there  would  be  no  reversal  If  Initial  cost 
differences  are  more  than  17.2X. 
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t»BLE  S 


Contractor/Oovernaent  Costs  - % Difference 


Z 

X 

X 

1 

d X 

t> 

2.33  X 3.76  « 

8.73 

4.7 

99* 

1.645  X 3.7S  « 

6.17 

7.3 

19.6 

9SX 

1.28  X 3.75  « 

4.8 

8.6 

18.2 

90* 

1.0  X 3.75  « 

3.75 

9.7 

17.2 

84* 

knottier  mlysls  can  be  ude  by  gnuping  the  s«p1ed  cost  conparlsons  by  total 
govenwent  costs  to  see  If  there  Is  any  st^itflcance  betaoen  the  nuater  of  observed 
reversals  and  the  govemaent  costs.  Three  categories  can  be  separated  - over  i2.000.000 
(17):  $500,000  to  $2,000,000  (14;  and  wider  $500,000  (12).  It  Is  Interesting  to  note 

that  7 of  the  observed  reversals  fell  into  the  over  $2,000,000  category;  2 vere  in  the 
IKOO.OOO^to  $2,000,000  category:  and  1 was  In  the  wider $500,000  • category.  This  observa- 
tion  Twds  to  the  inference  that  while  the  nunber  of  actual  reversals  to  be  emected  nay 
be  SUIT,  they  wey  occur  in  the  higher  cost  CITA  functions.  In  fact,  in  the  sanle.  whm 
govern^t  wts  on  all  43  cost  cowpaHsons  tottalled  $187,851,143.  a total  of  $76,697,646 
or  41*  would  actually  have  been  contracted  out  using  the  revised  rates.  The  greater  nwdwr 
of  revenals  in  high  cost  CITA  finctions  night  be  explained  by  the  fact  that  contractors 
subnitting  proposals  for  larger  contracts  have  subjected  their  estinates  to  closer  scrutiny 
since  they  obviously  would  like  to  get  the  larger  contracts.  This  also  ni*t  suggest  that 
rantractor  proposals  used  in  cost  co^iarisons  of  less  costly  CITA  functions  night  be  only 
token  bids  in  sone  instances. 


VALIDITT  OF  WBI  RATES 

Since  the  revised  civilian  benefit  rates  have  been  determined  to  have  a sioiificant 
co^arisons.  an  exanination  as  to  the  validity  of  the  new  rates  is  in 
analysis  of  the  way  the  new  rates  were  devised  by  the  Civil  Service  Cowission 
ws  published  in  a 23  August  1975  nenorandua  from  Hugh  Hitt,  then  Administrator  for  the 
OTfice  of  Federal  Procurement  Poliqr.l2  The  nemorandiai  also  requested  interested  parties 
to  subarit  their  victo  on  the  new  rates  by  September  20,  1976. 


Representatives  Morris  Udall  and  Oiristopher  Dodd  have  requested  a General  Accounting 
Office  investigation  of  the  new  rates'.iJ  however,  the  GAO  has  not  disputed  the  new  rates 
at  this  writing.  Neither  have  any  of  the  other  opponents  of  the  proposed  rates  offered  anv 
concrete  proof  as  to  their  invalidity. 

While  there  has  been  some  opposition  to  the  proposed  rates  by  the  two  major  government 
S?*  unions,  the  vast  roajoHty  of  over  50  comments  received  by  the  OFPP  as  of  Septeidier 
20,  1976  have  been  favorable. >4  it  i$  a well  accepted  fact  that  the  old  rate  of  8.44X  was 


c . Federal  Procurement  Policy,  "Cost  Comparisons  Under  0MB  Circular  A-76." 

Federal  tegister,  August  23,  1976,  p.  35581. 

Hershaw,  "Contract-Out  Probe  on  Way,"  Federal  Times.  Septeaher  20,  1976. 

I gO.  P* I • 

14  Contract  Services:  Most  COmnents  Favor  Proposed  OW  A-76  Supplenent  Raising 
Government  Retirement  Cost  Factors,"  Federal  Contracts  Report.  October  4,  1976,  647,  p.A-20. 
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This  rescardi  p«p«r  has  show  that  thera  wald  have  beat  a slgatflcmt  m^er  of 
reversals  of  past  U.  S.  Anv  cost  co^Miisoos  ehai  revised  civilian  c^>1oyeee  bmflt  rates 
were  used,  thus  proving  the  hypothesis.  The  fact  that  the  total  costs  of  10  out  of  43 
cost  co^rlsons  (23»)  reversed  In  favor  of  contracting-out  provides  a bnis  ftor 
Inferring  that  as  as  5S  out  of  240  CITA  fincttons  to  be  reviewed  next  fiscal  year  elAt 
be  c^acted-out.*»  It  has  also  been  show  that  idille  23X  of  the  In-house  versus  contract- 
"V  be  reversed,  these  reversals  tend  to  be  centered  In  the  higher  cMt  CITA 
fun^ons.  ^ nay  affect  as  wch  as  of  the  precurenent  dollar  currently  spent  on  ^ 
in-lious6  p®rfbr“" ' ■ ■ 


4.  *4  fl*®  • •*»<»  deternining  uMch  QTA  functions  presartly 

MMfled  for  In-house  perfornanoe  by  reason  of  being  less  costly  to  tho  goveriMt  will 
be  changM  to  perforwnce  by  the  private  secior  idwi  the  revised  rates  are  applied.  Results 

govemnent  and  contractor  costs  are  within  13.2» 
S*aurtmtcd  out”  that  future  cost  caparisons  idilch  cone  p for  review  will 


It  ts  recoBBwided  that  federal  agencies  faced  with  the  problw  of  Idcntlfylag  five 
rWMm  currently  perforaed  In-house  to  be  contracted-out.  utilize  thj  results  of  this 
study  to  deternine  idilch  factions  will  revert  to  private  sector  perfornanoe  whw  the  new 
"*"*I?*«**5  are  appl1ed.l6  By  Identifying  these  functions,  which  will  be  contracted-out 
?!i^  reviewed,  the  agency  can  thus  prevent  Inappropiiately  selecting 
a futi^OT  Wich  should  rewln  In-house  on  the  basis  of  cost.  The  waUsIs  shows  that 
by  MiMtIvely  reviewing  current  cost  coparlsons  with  differences  betwien  govcnawit  md 
mtra^r  onto  within  4,7*.  7.3*.  8.6*.  and  9.71,  there  will  be  a probability  of  99*. 
95X,  90t,  and  84X  respectively,  that  the  govensaent  cost  will  be  hl^r  when  the  acw  rates 

drC  IB€d. 

« mJLl’  fecowended  that  further  research  Into  cost  conparlsons  exaerines  a saaple 

saple  used  In  this  study  is  really 
repr^entatlve  of  the  U-  S.  Ar^  cost  co^)er1sons.  Future  research  can  also  be  conducted 
on  o^r  pi^ed  rate  changes  will  be  In  the  areas  of  Inflationary  cost  Increases  on 
Mteiials,  nllltaiy  wage  Increases,  Interest,  and  taxes. 


assuaes  a constant  yearly  nWber  since  FT  75. 


FT  75  figures  publish- 


ed In  ASD  (li  Nenorandua  August  27,  1976.]r* 
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Contractor  Cost 
^^o'vmDcnt  Cost 
Clrillin  rcriniisal 
Cost 

Nss  ClTillaa 


l,373.9ia 

1*023.974 


1.039.639  1.090.294  1.143.415 
1.006.075  1,055.021  1,106,350 


662,725  695.861  730,854  767,187 


1*  J ■ . ' * 


Personnel  Cost 

Jlev  Goeensient  Cost 

786.543 

1.147.792 

825,870 

1.136.084 

867,163  - 
1.191.530 

910.522 

1.249.685 

Ibtsis  (4  yrs) 

OoMxaecor 

Coat 

4.647,266 

’sJ 

OU  Gtmmnm 

knc  Coat 

4.191.420 

: 1 

1 __  « • 

Fort  Bcanin^ 

■■r  CorenncnC  Co.C  4.725,091 

7 813  Hm.  Ball  Eqalfaaat 

0 

Contractor  Cost 
CoTcmnent  Cost 

ymmi 

385,805 

255,924 

2d  year 
313.940 
249.852 

3d  year 
313.940 
249.852 

4th  yaar 
3U,940 
249,852 

, ‘ * 

0 

„ Co.*  176,429 

Bev  Cl.lllaii  PerMoael 

176,429 

176,429 

176,429 

Cost 

Rsw  Goverment  Cost 

209.391 

288.886 

209,391 

282.814 

209,391 

282.814 

209,391 

282.814 

\ / 

Total.  (4  yr.) 

Contractor 

Coat 

1,327,625 

0 

Old  Cosarua 

ant  Coat 

1,005.480 

■w  Gowemeac  Coat 

1.137.328 

. 

0 

o 

o 

C) 


I^Oata  gathered  fnw  TRADOC  files  courtesy  of  19r.  Paul  Erickson.  TRAOOC  CITA 
coordinator. 
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3,  Fort  LeeveoMorth 

S 710 

luect  and  Rodent  Control  • 

u 

-iK  •ija-  ii 

i-,  ' ■' 

1st  year 

2d  year 

3d  year 

4ch  year 

.Cootrector  Coet 

':ri39.436 

123,304 

129.469 

135,942 

'' ' ' Covemneot  Cost 

90,940 

93,353 

, 98,020 

102,922 

. . , ClvUlsa  Personnel 

‘ 72,100 

' Cost 

75,705 

79,490 

83,465 

. Hew  ClvlUen 
' Personnel  Cost 

1 

1 - 

U 

85.571 

89,849 

94,341 

99,059 

••^•-ftew  Covtmwent  CoeC 

-104.411 

107.497 

112.871 

118.516 

Totsls  (4  yrs) 

Contractor  Cost 

528.151 

U 

i • , 

Cld  Govemnent  Cost 
Mew  Covemneot  Cost 

385.235 

443,295 

4,  Fort  Leavenworth 

S 712 

1 • 

Befuse  CoUectlen 

A .T 

1st  year 

2d  year 

3d  year 

4t]i  year 

] ' .•  Contractor  Cost 

167,823 

155,451 

161,840 

168,549 

Covemmenc  Cost 

127,767 

131,140 

134,720 

138,516 

ClvUien  Personnel 

Cost 

Hew  Civilian 

• 77,723  • 

81,619 

85,670 

89.985 

‘ 

, ^ . Personnel  Cost 

92,244 

96,868 

101.676 

106,797 

vj 

■■"•  Hew  Govemnent  Cost 

142.288 

146.389 

150.725 

155.328 

Totals  (4  yrs) 

Contractor  Cost 

Old  Govemnent  Cost 

653,663 

532.143 

5.  Fort  Loavemiorth 


Rev  Government  Coat  594,730 
S 716  Motor  Vehicle  Operatlonc 


V . 1 

• 

1st  year 

■ 2d  year 

3d  year 

4th  year 

Contractor  Cost 

705,957 

741,254 

778,316 

. 817,232 

Govemaent  Cost 

552,241 

507,874 

553,267 

559,930 

\ .) 

Civilian  Peraonnel 
Cost 

237,722 

249,608 

262,088 

275,192 

l . • 

New  Civilian 
Personnel  Cost 

282,136 

296,243 

3U.0S4 

326,607 

Hew  Govemnent  Cost 

596.655 

554.509 

602. 2n 

611.345 

( 

Totals  (4  yrs) 

Contractor  Cost 

Old  Government  Cost 

3,042,759 

2,173,312 

2,364,742 
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6.  Fort  Leoremrarth 


•Jvi 


253,859 

144,868 


re-f't  ret**,  ii 

^ ^ Contractor  Cost 
Government  Cost 
• i. «- Civilian  Personnel 

Hew  Civilian  ' 

Persoirael  Cost  134,288 

Hev  Covemaent  Coat  166.008  

— — — Totals  (4  jrrs)  Contractor  Cost  - 
t.'  Covernnent  Cost 

. :'Kem  Government  Cost 

7.  Fort  Picket  , J 506 


S 717  Motor  Vehicle  Maintenance  " ^ 

1st  year  ^ 2d  year  Jd  year  4th  year 


264,244  275,074  288,828 
149.631  ' 157,112  164,968 


„ 113, 149  i 118,807 


141,004 

m^sau 


124,746  ^ * 130,984 

148,052  *155,456 

180^^418 


1,081,985 

' 616,579  ‘ 
707,694 


-Sfi  ft.*: 

Contrac ter  Cost  .414,083 

•’^■’  Government  Cost  217,887 

• . Civilian  PeraoiiDel 

Coat  156,775 

^ ' ’i » Hei#  Civilian 

Personnel  Cost  186,065 


^ Koo-Combat  Vehicle  Sunport  , 
1st  year  ^ 2d  year  3d  year  4th  year 


ri' 
V 


345,447 

212,598 

156,775 

186,065 


345,447  345,447 

212,598  . 212,598 


156,775 

186,065 


156,775 

186,065 


U 

O 

O 

o 

o 

o 

o 

o 

o 


— Totals  (4  yrs) 

Contractor  Cost 

1,450,424 

Old  Govemoent  Cost 

855,681 

u 

• i • 

New  Covernnent  Cost 

972,841 

Fort  Sill 

• , * • ^ . V* 

J 512 

Amament 

■ - i 

C) 

1st  year 

2d  year 

3d  year 

4th  year 

Contaactor  Cost 

381,845 

319,230 

319,230 

319,230 

U 

Goveriment  Cbst 

277,478 

:T277,472 

277,472 

877,472 

Civilian  Personnel 



Cost 

265,737 

265,737 

265.737 

265,737 

(_) 

New  Civilian 

Personnel  Cost 

315,385 

315,385 

315,385 

315,385 

New  Covernnent  Cost 

327.120 

327.120 

327.120 

327.120 

( ) 

Totals  (A  yrs) 

Contractor  Cost 

1,339,535 

Old  Covernnent  Cost 

1,109,888 

New  Covemoent  Coot 

1,308,480 

(“) 

\ 


'r  ^ - - •*  - f 


t h 
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I'  3'v 

-f 

If'  r 


I*  <> 
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i 


I 


li 


Contractor  Coat 
*'"  Covemocnt  Coat 
^ Civilian  Peraomel 
it--  Coot 

How  Civilian 
Poraonnal  Coot 
Covermont  Cost 


Tbtala  (4  vro) 

13*  Fort  box 

f ■ _ 1 . 

r '.-r  .-’V-'  ^c^v  i:=.  - 

Contractor  Cost 
Coverraieat  Coat 
ClTlllaa  Peraonnel 
Coat 

. Mao  ClTlllsn 
Paraonnel  Coat 
Hew  Coverreaent  Cost 


169,223 

134,474 

134,474 

100,229 

100,229 

100,229 

95,796 

95,796’ 

95,796 

113,693 

113,693  ■ 

113,693 

U8.127 

118.127 

118.127 

' Old  cGove 
■Haw  Cove 

X 939 


coa. 
it  Coat 
it  Coat 


572,655 
600,916  . 
♦72,507 


113,693 

118.127 


■ Bock  Quarry  Operation 


Totals  (4  yrs) 


lat  year 

2d  year 

3d  year 

4eh  year 

488,804 

474,261 

874,261 

474.261 

585,583 

324,647 

324,647 

324.647 

145,844 

145,844 

145,844 

U5,844 

173,092 

173,092 

173,092 

173,092 

612.831 

351.895 

351.895 

351.895 

Old  Govemnent  Cost 
Hew  Govemaent  Cost 


1,559,524 

1,668,516 


14.  Fort  Lee 


V 824  and  « 826 


Prograwnlng  and  Date 
Processing 


' ■ . 1 • 

1st  year 

2d  year 

3d  year 

4th  year 

Contractor  Coat 

769,359 

8U,828 

856,641 

903,927 

Govcmant  Coat 

743,450 

738.723 

779,501 

822.530 

! Civilian  Peraonnel 

) Coat 

324,392 

342,298 

361,193 

381,131 

1 Hew  Civilian 

Peraonnel  Coat 

384,999 

406,250 

428,675 

452,838 

New  Government  Cost 

804.056 

802.675 

846.983 

893.737 

Totals  (4  yra) 

Contractor  Cost 

3,341,755 

- 

Old  Govemaent  Coat 

3,084,204 

New  Govemaent  Coat 

3,347,451 

731 


o 

o 

o 

o 

o 

o 

o 

o 

O ‘ 

'i 

O ! 


\ 


-ji  fi  I . 1 PM  f.  . .1  u 


■■  i 


k 


I."  -:4’ 


* ‘ 


1S«  Fort  Bolvlor 


;■•  ^',v  —i-  ' 1.  /■•  ■ 

;--.r-'r,:. 


V 824  and  V 826  Prograonlng  and  Data 
Proceaalng 


- 

J • , ' 

.. . ._  ,. 

>■ 

:o: 

- 

lat  year 

2d  year 

3d  year 

4th  year 

^ , 

Contractor  Coat 

*'  949.158 

943,926 

991,123 

1,040,679 

/ V 

- Covetniwnt  Coat  * 

441,284 

463,348 

. 486,517 

510,843 

Civilian  Pareonnal 

* 

• > 

.Coat 

. 195,573 

205.352 

215,620 

226,401 

•o 

Rev  CivUlaa 

Personnel  Coat 

232.112 

243.718 

255.905 

268.700 

477.823 

501.714 

526.802 

553.142 

*"  Totals  (4  yra) 

Contractor  Cose 

3,924.686  7 . 

( ',  • 

l,901.9n: 

• 

nt  Cost 

2.059,481 

y 

v-.V;"  ' 

.u  ■ 

-*  ^ . -1  T m » • ^ 

. . _ ► : 

- 1.  • 

" 

k.‘  . 

V-  ‘ * .-7  * %'  • 

■ ■ f • 

j ^ 

'• 


'U 

(. 
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t.f.  TT. 

• DARCOH  COST  0(m.*ARISOII  SA>IPU 
Saratma  Any  Depot  S 724  ‘ 'ouatd  Servlcea 

lac  7«ar  2d  year  3d  year  _ 4t&  year 


Cofitrsctor  Cost 

1,155,605 

1,021,408 

1,072,478 

1,126,101 

Govenuaent  Cost 

432,239 

453,849 

476,541 

500,368 

CivUlsii  Persofioel 

Cost 

Nev  Civlllin 

352,834 

370,475 

388,999 

403,449 

Persoimel  Cost 

418^54 

429,691 

461,676 

484,760 

New  CovemDent  Cost 

498.159 

523e065 

549.218 

576.679 

ToulIs  (4  yrs) 

Contractor  Cost 

4,375,5  2 

• 

Old  Government  Cost 
Nev  Governnemt  Cost 

1,862,997-: 

2.147.121 

Ssraxnui  Any  Depot 

Z 945 

' Machine 

Shop  Operation 

Isc  year 

2d  year 

5d  year 

4ch  year  ■ 

Contractor  tost 

838.045 

4:4S6,«09 

7i4.208 

456.609 

714,208 

456,609 

714,203 

Civilian  Personnel 

456,609 

Cost 

Hev  Civilian 

361.452 

361.452 

361,452 

361,452 

Personnel  Cost 

428.982 

428.982 

428,982 

428.982 

New  Covernnent  Cost 

524.140 

S24.140 

574.140 

524.140 

Totals  (4  yrs) 

Contractor  Cost 

2,^60.669 

• ) 


Old  Govenmnt  Cost 
New  Covemzaent  Cost 


1,826,436 

2,096,558 


l;  . 

. i 

I 

L) 

C) 


I 13 


coordinator^  gathered  form  DARCOM  files  courtesy  of  Mrs.  Aina  Weaver,  DARCOH  CITA 
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3«  SUcraxaento  Army  Depot 


S 724  Guard  Serricea 


-i  -f.-)  - fX 

1st  year 

2d  year 

3d  year 

4th  year 

Contractor  Cost 

5ZI,729 

516,361 

541,256 

567,395 

Goremment  Cost 

480,959 

504,682 

529,666 

555,899 

Civilian  Personnel  * 

Cost 

. 423,641 

445,033 

467,285 

490,649 

Nev  Civilian 

• 

• 

Porsonnel  Cost 

503,028 

528,179 

554,589 

582,318 

Kev  Goverment  Cost 

560.146 

587.828 

616.969 

. 647.568 

.Totals  (4  yrs) 

Contractor  Cost 

2,152,741 

--  - 

♦ 

Old  Govemnent  Coot 

2,071,206 

• ' 

V \ iii 

Kev  Government  Cost 

2,412, 5U 

4.  Sacramento  Army  Depot 


, # i ‘ ‘ 


T 601  Packing  and  Ctatlng 


5. 


••  ’ r ^ . j/f 

let  year 

2d  year 

3d  year 

4th  year 

Contractor  Cost 
GoveriBient  Cost 
Civilian  Personnel 

6,290.640 

3,641,092 

5,297,610 

3,823,239 

5,562,491 

4,014,413 

5,840,616 

4.215.133 

Cost 

2,452,219 

2,.t74,830 

2,703,572 

2,838,751 

Rev  Civilian 

Personnel  Cost 

2,910,370 

3,055,889 

3,208,684 

3,369,119 

Rev  Govemnent  Cost 

4.099.243 

4.304.308 

4.519.525 

4.745.501 

Totals  (4  yrs) 

Contractor  Cost 

22.992,357 

.• 

Old  Government  Cost 

15,693.887 

^ , 

Nev  Government  Coot 

17,668,576 

Tim  Proving  Ground 

S 724  Ooard  Services 

‘ 

1st  year 

2d  year 

3d  year 

4th  year 

Contractor  Cost 

866,462 

966,582 

1,014,911 

1,065,656 

Goverment  Cost 
Civilian  Personnel 

684,472 

720,410 

756,430 

794,252 

Cost 

151,734 

159,320 

167,286 

175,650 

Rev  Civilian 

Personnel  Cost 

180,083 

189,086 

198,540 

208,467 

Nev  Govemnent  Cost 

712.821 

750.176 

737.684 

827.069 

Totals  (4  yrs) 

Contractor 

Cost 

3.913,611 

Old  Covernioent  Cost 
New  Covernocnt  Cost 


2,925, 56« 
3,077,750 


734 


8. 


735 


u 

O 


ledstcme  Arsenal 

T 601 

Packing 

and  Gracing 

lac  year 

28  year 

3d  year 

4ch  year 

o 

Contractor  Cost 

166,264 

166.264 

166,264 

166,264 

Government  Cost 

101.313 

101.313 

101,313 

101,313 

Civilian  Personnel  * 

Kj 

Cost 

96,177 

96.177 

96,177 

96,177 

Kev  Civlltan 

Personnel  Cost 

U4,146 

114,146 

114,146 

114,146 

u 

Kev  Government  Cost 

119.282 

119.282 

119.282 

119.282 

Totals  (4  yro) 

Contractor  Cost 

665,056 

Old  Govexm 

■enc  CosC 

405,252 

u 

Mew  Govennent  CosC 

477,128 

Kedscooe  Arsenal 

V ^ *■ 

T 814 

Itiellng 

Service 

o 

u 

laC  year 

2d  year 

3d  year 

4ch  year 

Contractor  Cost 

68,945 

68,945 

68,945 

68,945 

o 

Government  Cost  ' 

41.243 

41,243 

41,243 

dvilian  Personnel 

Cost 

39,096 

39,096 

39,096 

39,096 

u 

Cev  Civilian 

. 

Personnel  Cost 

46,400 

46,400 

46.400 

46.400 

Kew  Government  Cost 

48.547 

48.547 

48.547 

48.547 

4 \ 

Totals  (4  yrs) 

Contractor  Cost 

275,780 

u 

Old  Covemmett  Cost 

164,972 

New  Government  Cost 

194,109 

Redstone  Arsenal 

T 805 

Operation  of  Bulk  Liejuid  Storage 

• 

lac  year 

2d  year 

3d  year 

4ch  year 

o 

Contractor • Cost 

79,908 

79,908 

79,908 

79,908 

Government  Cost 

45,651 

45,651 

45,651 

45.651 

Civilian  Personnel 

{ ) 

Cost 

43,418 

43,418 

43.4ia 

43,418 

llev  Civilian 

Personnel  Cost 

51,530 

51,530 

• 51,530 

51,530 

L) 

Kev  Govemnent  Cost 

53.763 

53.763 

53.763 

53.763 

Totals  (4  yrs) 

Contractor  Cost 

319,632 

Old  Covemaent  Cost 

182,604 

1 ) 

New  Government  Coat 

215,052 

\ / 

! 


ij 

o 

C) 


WMiii  liiniiim  nil  ■ii"i<i»'i*i 
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9.-  DARCOH  Auto.  Log.  Mgt.  Sy«.  Agency  W 826  ProgrsHdng  Services 

”■  t , 

, yewr  2d  year  3d  year  6th  year 


•>7  Contractor  Cost 
Gorvenment  Cost 
Civilian  Personnel 
Cost 

Nev  Civilian 

Personnel  Cost  9,272,312  9,735,928  10,222,726  10,733,861 

. Bev  Coverwaept  Cost  10,738,722  11.275.659  11,839.661  12.631.613 


20,355,859  19,111,873  20,067,666  21,070,839 
9,279,072  9,763,026  10,230,177  10,761,685 

7,812.662  8,203.295  8,613,660  9,066,133 


Totals  (6  yrs)  Contxaetor  Cost  80,606,037 

Old  Govemaeot  Cost  39,993,960 
Mew  Covenaent  Cost  66,285,235 


. - V _ - 

10.  OAKCOM  Auto.  Log.  Mgt.  Sys.  Agency 


V 826  Data  Processing 


. . ' w.  ... 

1st  year 

2d  year 

3d  year 

4th  year 

Contractor  Cost 
Covemnent  Cost 
Civilian  Personnsl 

4,197,864 

1,630.083 

4,233,399 

1,362,801 

6,447,695 

1,430,943 

6,670,956 

1,502,489 

Cost 

* Bev  Civilian 

626.427 

657.748 

690,635 

725,167 

'..A 

Personnel  Cost 

Hew  CoveriTjent  Cost 

743,463 

-J.W.119 

780.636 

1.485.689 

819,667 

1.559.975 

860,651 

1.637.973 

/ 5 . 

Totals  (4  yrs) 

Contractor  Cost 

17,549,912 

5,926,316 
New  Covemaent  Cost  6,430,756 

11.  Coinpus  Christl  Amy  Depot  S 717  Motor  Vehicle  Maintenance 


. 

1st  year 

2d  year 

3d  year 

4th  year 

V ■ ■ 

Contractor  Cost 
Covemnent  Cost 
Civilian  Personnel 

176,255 

133,729 

157,873 

133,729 

157,873 

133,729 

157,873 

133,729 

Cost 

Hew  Civilian  Personnel 

83,136 

83,136 

83,136 

83,136 

i • 

Cost 

Kew  Covcrrrient  Cost 

98.668 

149.261 

98,668 

149.261 

98,668 

149.261 

. 98,668 
149.261 

Old  Covemnent  Cost 
New  Covemnent  Cost 


736 


649,876 

536,916 

597,046 


A 


t =. 


I 


12.  Corpus  Christ!  Amy  Depot 


Contractor  Cost 
Govemnent  Cost 
Civilian  Personnel 
^ Coot 

Rev  Civilian 
'**  Ittrsonnel  Cost 


1st  ykar 

816, 93S 
634,405 


T 801  Packing  and  Crating 

. t - . 

M year  3d' year  ‘ 4th  year 


760,935 

439,186 


760,935  760,935 

440,402  , 439,836 


185,406  ■ 185,406 


220,046 


->  Ww  Covemment  Cost  469*nAS 

• /X \ " ■ ■ 


tetals  (4  yrs} 

13.  Edgeuood  Arsenal 

Contractor  Cost 
' ‘ Govemnent  Cost 
Civilian  Personnel 
Cost 

Rev  Civilian 
* ' • Personnel  Cost 


220,046 

473.826 


CoBtractor  Coot 

Old  Govemnent  Cost 
Rev  Govemnent  Cost 


185,406 

22M46 

4751042 


185,406 

220,046 


3,^IM 

1*753,829 

1,892.389 


474.476 


W 826  Progrannlng  Services 
1st  year  2d  year  ’ 3d  ye» 


P f \ 


254,117 

124,828 

117,690 

139,678 


210,259 

124,828 

UI,690 

139,678 


210,259 

124,828 

117.690 

139,678 


j Totals  (4  yrs} 

Contractor  Cost 

Old.iCovemswnt  Cost 

Rev  Govemaent  Cost 

884.894 

499,312 

587.265 

• Edgevood  Arsenal 

W 827  Dete  Processing 

Ist  year  2d  year 

3d  year 

Ccntractor  Cost 
Govemaent  Cost 
Civilian  Personnel 
Cost 

Hew  Civilian 
Personnel  Cost 

IM  Covemment  Cost 

116,612  113,079 
80,795  74,645 

58,159  58,159 

69,025  69,025 

. 91.661  85.511 

113,079 

74,645 

58,159 

69,025 

85.511 

Totals  (3  yrs) 

A 

Contractor  Cost 

Old  Govemoent  Coat 

Mew  Govcnssent  Cost 

342,770 

230,035 

262,682 

' — V-f  fti 

4th  year 

210,259 

124,828 

117,690 

U9.678 

146.816 


737 


U 

O 

o 

o 

o 

o 

^1 

o- 

o 

o 

c.) 

o 

o 

C) 
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o 
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15*  Letterkenny  Array  Depot 


S 716  Hotor  Vehicle  Operation 


Ist  year  2d  year  3d  year 


. 'Contractor  Coat 
Government  Coat 


828,329 

648,554 


813,947 

680,981 


854,644  897,377 

715,030  , 750,782 


t-  ^ Civilian  Personnel 
‘ Cost 
. ^ Kev  Civilian 
- N Peraonnel  Coat 
^'Hev  Covemaent  Co^ 
t.*  Totals  (4  yrsi^ 


458,667  481,600 


544,360 


571,578 

770.959 


505.680  530.964 

f 

600,157.  630.165 

869.983 


tCor  Cose 
Old  Govenaent  Cost 
Hew  Covenoent  Cost 


809. 50T 

3,394,297 

2,795,347 

3.164,696 


16.  Letterkemy  Amy  Depot 

’-tj.’  /•*  ' . 


1st 


u 

f » 

Contractor  Cost 

5*694.043 

4.944.945 

4,944,945 

4,944,945 

* Governnent  Coat 

4,914,140 

4,914,140 

4,914,140 

4,914,140 

Civilian  Personnel 

Coot 

3,913,221 

3,913.221 

3,913,221 

3,913,221 

, Mm  Civilian 

. 

Personnel  Coat 

4,644,333 

4,644.333 

4,644,333 

4.644.333 

( 

^ Govemnnnt  Coat 

5.645.252 

5.645.252 

5.645.252 

5.645.252 

V.  ' 

Totals  (4  yrs) 

Contractor  Cost 

20,528,878 

Old  Govemaent  Cost 

19,656,560 

( 

Nev  Govemnent  Cost 

22,581,010 

17,  Letterkenny  Amy  Depot 

T 817  Other  Uoo-Mfg. 

i 

Opns. 

L 

► 

1st  year 

2d  year 

3d  year 

4eh  year 

Contractor  Coat 

6,310.149 

6,310,149 

6,310,149 

6.310,149 

Governnent  Cost 

5,483,942 

5,483,942 

5,483,942 

5,483,942 

Civilian  Peraoianel 

Cost 

4,549,864 

4.549,864 

4,549.864 

4.549.864 

/ 

!lev  Civilian 

( 

. Personnel  Cost 

5,399,921 

5,399,921 

5,399,921 

5,399.912 

Nev  Governnent  Coat 

6.333.999 

6.333.999 

6.333.999 

6.333.999 

Totals  (4  yrs) 

Contractor  Cost 

25,240,596 

(_ 

Old  Govemomt  Cost 

21,935,768 

. 

Kev  GoveroBent  Cost 

25,335,997 

T 801  Tsekiog  sttl  Crating 


2d  year  3d  year  4th  year 


738. 


\ 


-? 


19. 


u 

Plcatinny  Arsenal 

1 

T 812  Rail  Transput  Services 

.0 

,1  V 1 •-  . J^.  ■_ 

1st  year 

1 

2d  yeer 

3d  year 

’ . * « r-  - 

4th  year 

o 

Contractor  Cost 
^ Covemocnt  Cost 
■'  Clrlllsn  Personnel 

56,430 

47,327 

a 

56,430 

47,327 

56.430 

47,327 

56,430 

47,327 

Cost 

Hev  Civilian 

45,253 

65,253 

45,253 

65,253 

u 

Bersoimel  Cost 
Bbw  Govemaent  Cost 

53,708 

55.781 

53,708 
- 55.781 

53,708 

55.781 

53,708 

55  781 

0 

■ Totals  (4  yra) 

Contmctoi 

‘ Cost 

225,720 

^ ><_  .*  *1  •- 

4 • "• 

Old  Covemnent  Cost 

■a*  Covemnent  Cost 

*“  ■ • 

189,308 

223,127 

O 

'•  Plcatimi^  Arsenal 

X 931  Ordnance  Equipment 

*’r  ^ ».■  ■ 

let  yeer 

2d  year 

3d  year 

6th  year 

o. 

Contractor  Coot 
Goveranent  Coot 
Civilian  Personnel 

3,276,827 

2,433,393 

2,523,111 

2,433,393 

2,523.U1 

2,433,393 

2,523.1U 

2,633.393 

0 

Cost 

^ Heir  Civilian 

2,316,605 

2,316,M5 

2,316.605 

2.316,605 

T) 

Personnel  Cost 

Kew  Covemnent  Cost 

2,749,420 

2.866.207 

2,749,420 

2.866,207 

2,749,420 

2.866.207 

2.769,620 

2.866.207 

- . Totals  (4  yra) 

Contractor 

Coat 

10,846,160 

1 \ 

Old  Government  Cost 

9,733,572 

Kev  Covemnent  Cost 

11,464,830 

Picatlnnj  Arssnal 

T 801  Packing  and  Crating 

O 

1st  yeer 

2d  year 

3d  year 

4th  year 

Contractor  Coat 
Covemnent  Cost 

Civil tan  Personnel 

238,269 

155,139 

168,051 

155,139 

168,051 

155,139 

168,051 

155,139 

Cost 

Nev  Civilian 

148,797 

148,797 

148,797 

148^797 

Personnel  Cost 

Hev  Coven ifoent  Cos^ 

176,597 

182.939 

176,597 

182.939 

176,597 

182.939 

176,597 

182  930 

0 

Totals  (4  yra) 

Contractor  Coat 

742,422 

Old  Covemnent  Cost 

620,556 

• 

Kev  Covemnent  Cost 

731,756 

, 

l_) 

• 

739 

U 

0 

• 
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24.  Watcrvllet  Arsenal 


S 716  7lotor  Vehicle  Operaclons  . v i 


, i . l«>ur  i 

f ' Contractor  Cost 
<.  Covemaent  Cost 
ClvlUan  Personnel 
. .i  c<»t 

Kcv  Civilian 
se  t <.  Personnel  Cost 
jfctt.Covemwent  Cost 
Totals  (3  yrs) 

w I I i*".  ^ -1'  •; 


1st  year 

2d  year 

639,623 

652,880 

239,152 

205,726 

: 135,807 

• 147,597 

161,180 

175,173 

264.525 

233.302 

Contractor  Cost 

Old  Covemaent  Cost 
Hew  Covemaent  Cost 


3d  year  4th  year 

685,524 

215,281 

149,727  ’ 

177,701 
243.181 
1,978,021  ^ 

660,159 
841,018 


25.  Ustervlist  Arsenal  T 812  EsU  Itansport  Services  . 


Tear  t * 1st  year  • 2d  year  3d  year 


Contractor  Cost 
Covemwnt  Cost 
Civilian  Personnel 
Cost 

Hew  Civilian 
Personnel  Cost 
^-.  .Hcw  Covemaent  Cost 

> 140,521 
69,795 

66,347 

78,742 

82.191 

110,250 

73,284 

69,664 

82,679 
86.299 

115,763 

76,947 

73,147 

86,813 

90.611 

Totals  (3  yrs) 

Contractor  Cost 

366,534 

Old  Covemaent  Cost 

220,026 

Hew  Covemaent  Cost 

259,103 

4th  year 

•:tv7 

- ^J***  is* 

•1  /.  . 

. »‘T  ^ 


26.  Fort  Honaouth 


S 714  Fhmlture  Repair 


Contractor  Cost 
Covemaent  Cost 
Civilian  Personnel 
Cost 

Hew  Civilian 
Personnel  Cost 
Hew  Covemnent  Cost 

Totals  (4  yrs) 


1st  year 

2d  year 

104,055 

111,703 

88,682 

• 101,312 

73,937 

85,755 

87,751 

101,777 

102.496 

117.334 

Contractor  Cost 

Old  Covemaent  Cost 
Hew  Covemaent  Cost 


3d  year  4th  year 

1U,703  1U,703 

101,312  101,:i2 

85,755  85,755 

101,777  101,777 

_ 117.334  117. 33A 

439,164 

392,618 

454,497 
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27.  Savamui  Army  Depot 


l- 

i 

? 


1 i j ! r ! ! lit  jfrr 

i Contractor  Cost 
Cover.vacnt  Cost 
* Civilian  Personnel 
j icost 

New  Civilian  Personnel 
! Cott 

Itew  Covemaent  Cost 
Totals  (4  yrs) 


T 812  lailrioad  Operations 

2d  year  3d  year 


-'■yrr 

• it  ! 

4th  year 


I 


191,831 

179,661 

179,661 

436,381 

112,381 

112,381 

74,733 

74,733 

. 74,733 

88,695 

88,695 

88,695 

450,343 

126.343 

126.343 

♦ 

} 

28s 


Savanna  Axwj  Depot 


Contractor  Cost  730,814 

J . Government  Cose  773,524 

j New  Covemaent  Cost  829,372 


179,661 

112,381 

74,733 

f % • J 

88,695. 


■ -n 


.S  712  Refuse  Collection 


•rti;-;-: 


t * 

*1 

I*  r 


‘ 'i 

^ V 

' V »r 


Contractor  Cost 
Covement  Cost 
Civilian  Personnel 
Cost 

New  Civilian  Personnel 
Cost 

New  Covemnent  Cost 


^ Totals  (4  yrs) 

; i ■ . . ii  : 

( . f; 

. i 

I • • ' 


: year  • 

2d  year 

3d  year 

4tb  year 

73,707 

60,000 

60,000 

60,000 

79,200 

36,637 

36,637 

36,637 

31,375 

. 31,375 

31,375 

31,375 

37,236 

37,236 

37,236 

37,236 

«.062 

42.499 

42.499 

42.499 

Controctor  Cost 

Old  Covemaent 
New  Covemaent 


253,707 
189,  m 
212,559 


1 
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CHAPTER  I 
OVERVIEW 


^r  objective  Is  to  develop  a model  of  the  logistic  process  to  be  used  In  unocr-  / 

standing  the  flow  and  Interdependencies  therein.  Logistics  Is  a process  which  has  tht  ’ 

objectl^  of  supporting  an  operation  (7:  3-4).  l-e  reviewed  many  definitions  of  logls- 
itcs  wd  Its  relaUd  functions  and  processes  In  an  attempt  to  place  the  development 

•»3»stic  proce;'  model  In  perspective.  However,  definitions  found  did  not  pro-  I 
Tw*  abstraction  nec  ssary  to  construct  a truly  representative 

mo«l.  Therefore,  t*»e  following  definitions  were  fomilated  based  on  oui  own  education 
and  experience: 

1.  L^lstlcs— a process  by  which  requirements  determination,  acquisition, 
allocation,  and  disposition  of  resources  (material,  manpower  a d money) 
can  be  effected  In  support  of  a joo. 

2.  Job— a definable  Indenture  of  a mission  In  which  resources  are  employed 
or  consuned. 

3.  Mission— the  reason  for  organizational  existence. 


Having  defined  the  starting  point  for  describing  the  logistic  process,  a frame- 
wrk  was  needed  to  display  process  interdependencies-  We  found  that  a general 
frammork  could  be  obtained  in  systems  theory  and  a specific  exp1ari«tory  perspecti/e 
and  level  of  detail  could  be  addressed  using  Black  Box  Tbeory.  As  typically  defined, 
a system  Is  a set  of  covponents  surrotxided  by  a boundarv  which  accepts  inputs  from 
some  other  system  and  discharges  outputs  Into  another  »tera  Systems  theory 

s cuncerned  with  problems  ot  relationships,  struc^ome,  and  Interdependence  rather 

constant  attributes  of  objects  (5:18).  Open  syt^  ms  theory  implies  tha**  ihe 
oj  a S3  cm  do  not  completely  determine  the  system**  outcomes  by  themselves,  but 
with  an  outside  environment  that  represents  situational  uncertainty 
|b:99).  The  parts  of  the  system  are  subject  to  In'^luence  by  envl  '•nmental  stimuli 
not  directly  contained  within  the  system. 
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Typically,  a systoi's  perfomance  is  more  sensitive  to  changes  In  certain 
components  than  others.  The  detection  and  manipulation  of  these  sensitivities  In 
establishing  priorities  for  changes  to  be  made  In  the  flow  through  the  system  Is 
referred  to  as  sei:s1t1v1ty  analysis.  System  performance  depends  on  the  balance  which 
Is  achieved  among  system  components,  measured  In  terms  of  capacity,  timing,  economics, 
and  other  measures.  Often,  performance  reflects  the  effectiveness  of  the  weakest  or 
least  efficient  element  In  the  system,  regardless  of  the  degree  of  technical  sophls- 
* ticatlon  or  capability  associated  with  other  elements  In  the  system.  Optimum  system  i 
‘performance  does  not  necessarily  depend  on:  (1)  the  optimun  performance  of  each 
Individual  system  component.  (2)  the  ability  to  cut  the  single  most  Important  cost 
Item  In  the  system  budget,  or  (3)  use  of  the  most  advanced  technology  for  system 
components.  The  management  of  systems  Involves  cost  tradeoff,  which  Is  a concept 
that  relies  on  the  belief  that  all  activities  and  results  can  be  translated  Into 
costs  whether  they  are  as  a result  of  activities,  e.g.  contracting  and  training,  or 
foregone  opportimitles,  e.g.  failure  to  perform  a job  and  then  abo<’ting  a mission 
H-22).  . - ."w,,-  - . ..  i, 

A systens  aodel  may  be  developed  by  integrating  the  functions/processes  of  various 
subsystems.  The  outputs  from  one  subsystem  are  simply  the  Inputs  to  another  subsystem 
via  a iQfrlad  of  cooplex  paths  or  Iterations.  Adequately  describing  such  a complex 
system  depends  on  the  use  to  which  the  description  Is  to  be  put  because  In  building 
systems  from  subsystems  and  in  decomposing  systems  Into  subsystems  the  task  may  be 
conceptually  Infinite.  In  this  respect  It  becomes  obvious  that,  at  some  point,  the 
effort  required  to  describe  the  next  subsystem  level  In  detail  Is  not  worth  the 
advantage  gained  In  doing  so.  At  this  point  the  subsystem  may  be  described  In  simple 
Input-output  terms  and  may  be  treated  as  a "black  box,"  I.e.  an  entity  which  does 
known  tnings  in  unknown  ways  with  unknown  mechanisms.  Among  the  reasons  why  one  may 
desire  to  apply  the  black  box  approach  are:  (1)  the  level  of  Inforwiatlon  needed 
concerning  the  system  may  be  only  superficial,  or  (2)  understanding  of  the  system  In 
question  may  be  so  poor  that  to  describe  It  In  more  specific  terms  would  be  counter- 
productive (2:46-47). 

(kir  approach  to  constructing  a model  of  the  logistic  process  was  also  Influenced 
by  several  considerations  and  characteristics  which  we  felt  were  desired  In  such 
a model: 


1.  The  model  should  be  constructed  to  minimize  bias  to  any  one  Interpretation 

• of  the  logistic  process. 

2.  The  level  of  model  abstraction  should  be  constructed  so  that  the  model  would 
address  any  indenture  of  the  logistic  process  or  a series  of  sequential 
indentures  In  the  logistics  process  in  an  Iterative  manner. 

3.  The  model  should  be  constructed  to  be  relevant  to  tne  logistic  process  as  It 
applies  both  to  the  commercial  and  railUary  sections  of  the  econooy  although, 
necessarily*  some  of  the  subprocesses  are  couched  In  military  terms. 

4.  Unnecessary  duplicative  subsystems  should  be  minimized  so  that  the  operation 
of  the  oodel  in  addressing  a specific  mission  could  be  understood  with 
relative  ease. 

5.  In  keeping  with  black  box  theory  the  detail  of  each  subroutine  should  be 
detailed  just  enough  to  allow  general  model  application  without  the  combina- 
tional problems  envisioned  In  addressing  every  possible  contingency. 
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6. 


The  mo^l  should  be  designed  so  that  at  any  point,  the  otodel  is  abu  tn 


CHAPTER  II 

REQUIREMENTS  DETERMINATION 


the  ass«5tion*tha? t^^ISion^iI^)5t*fro!^i^®enviro^^lSt"i??^  engaged  leider 

requimwts  detenaination  phasrKiJs  is  “* 

the  logistics  system.  ill-definS  Continu^'  ii[fo^Lf?«^“^  il*  * phase 

betneen  the  system’s  objective  andthe^ui^JfSIS^^U.M'^T 
preclude  the  omission  of  required  eleLTS  rfS^?!!*™'’"*!!®"  P*""  “ “ ^ 
possible  definition  of  -wSsSy  TSi^Jicf,JS^L!“PP?'^  ?:  * "!«  »•«»<* 

need  for  timely  information  fl»  is  J"  situation  the 

wouldoCoapound  the  objective  accomplishment  S «v1rafrJs5e°S?^1fi:  ^SSilce. 

paraStSj  Is  necessary  to  quantify  the 

be  a gross  unit  of  measure  of  a Jings  operation'^*With°Mfh^*^r"*®'^®*" 
resources  could  be  further  definmi  hw  sortie  the  units  of 

also  by  material  and  parts  r^IiJjLentrin  * 

of  resources  are  established  an  evaluation  sortie.  Once  discrete  units 

can  be  initiated  anda  determinati™  *7^  “"certainty  of  the  objectives 

“ “ ”*  fsinn.,  of  cunrg  the 

Or  possiH,.  the  ^ST™?  4 Urtl3leT''"J'"'  f™  O"'"*  « to  PoW  B. 

of  the  logistical  sSJrSt  Sue  to  mcJJ^vI  neces^y 

risk  and  uncertainty  analysis  evolves  arSnd  “SH"  fMsihTnl^*' 
decides  tc  continue  or  discoutinue  resnurm  *"**  management 

logistics  support  is  determined  iSfMsibirt^wt-  system’s  objective.  If 

Of  its  essentiality  and  eitte?  qiIera~atei^,X^-f‘”'*  **  ^evaluated  in  terms 
is  to  be  viable.  ^ ^ priority  or  adjusted  if  logistical  support 
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Figure  1.  Requirements  Detcr.'nination 
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As$uning  the  risk  and  uncertainty  Involved  in  supporting  the  objective  to  be 
acceptable*  the  task  then  becones  one  of  developing  and  analyzing  alternatives. 
Alternatives  are  developed  within  the  scope  of  accomplishing  the  objective  and 
Involve  the  various  combinations  of  men*  material  and  dollars  committed  to  the 
logistical  task.  Time  Indirectly  could  also  be  taken  Into  account  as  a resource* 
particularly  as  it  relates  to  scheduling  and  the  decision  of  commitment  of  other 
resources.  The  conmon  denominator  in  determining  alternatives  Is  “dollars."  Although 
it  is  often  difficult  to  identify  dollar  tradeoffs  elsewhere  In  the  logistical  system* 
the  alternative  evaluation  of  various  combinations  of  resource  requirements  is  reason* 
ably  approachable.  For  example.  In  a radar  assembly*  repair  alternatives  can  be 
formulated  around  the  various  cost  or  dollar  combinations  of  men  and  equipment. 
Alternatives  can  be  further  developed  to  a detail  involving  the  skill  level  of  the  men 
and  the  technical  sophistication  of  the  test  and  repair  equipment.  i 


!;id 


Once  the  manpower  skill  level  and  material  requirements  have  been  established*  the 
task  becomes  one  of  evaluating  the  utilization  basis  associated  with  the  men  and 
material  resources.  This^cision  is  based  on  an  evaluation  of  the  penalty  Incurred 
by  the  total  system  in  the  absence  of  the  resource  or  the  item  at  any  point  in  the  * 
system.  In  as  much  as  a circuit  breaker  in  an  electrical  system  is  seldom  required  to 
fwictioo  it  cannot  be  omitt^  of  the  potential  oenaltv  if  it  is  excluded  as 

a system  requirement.  Similarly*  the  basis  of  required  manpower  must  be  est^bnshe 
on  the  necessity  of  full-time  manning  (post-manning)  regardless  of  a sporadic  work 
load  or  on  average  manning  where  a potential  system  queue  would  not  per**ll2e  the 
systaa's  operation  excessively.  With  materials*  the  orientation  adjusts  slightly 
with  regard  to  the  basis  of  material  utilization.  Considerations  must  be  oiven  to  th^ 
cost  of  repairing  an  Item  (If  possible)  or  declaHng  the  item  consuS  In  thl  o^^ 
tion.  With  fuel*  for  example*  the  determination  Is  obvious  In  that  the  material  Is 
consimied.  However*  a generator*  once  failed  could  be  repaired  or  considered  consumed 
and  the  determination  would  most  likely  depend  on  the  Item's  criticality*  repair 
capability*  MTTR*  and  costs. 


As  requirements  are  categorized  on  the  basis  of  utilization  they  again  can  be 
quantified  In  terms  of  cost  or  "dollars."  The  specific  resource  requirements*  once 
delineated*  are  assuaed  to  be  the  most  logical  and  rational  list  of  requirements 
necessary  to  fulfill  the  desired  alternative.  However*  once  this  alternative 
Interacts  with  the  mainstream  of  the  logistical  network  and  associated  environmental 
factors,  the  potential  for  political  override  of  the  alternative  and  associated 
requirements  Is  readily  apparent. 


CHAPTER  III 

ACQUISITION*  ALLOCATION*  AND  DISPOSITION 

j 


The  purpose  of  this  chapter  Is  to  guide  the  reader  through  our  logistical  network. 
Some  areas  within  the  network  are  easily  comprehended  by  just  examining  the  flow 
chart.  Other  areas  require  a written  explanation  of  the  Interfaces  presented.  This 
chapter  Is  structured  to  explain  the  network  from  beginning  to  end  but  to  only 
elaborate  on  those  areas  requiring  extra  explanation. 


C’ 


L 


Dfecldinq  Wiat  Is  Needed  (Figure  2) 

' ' Vl  ' 

Logistics  provides  a service;  it  does  not  fight  a war,  for  example,  it  feeds 
resources  to  a war.  A logistics  network  must  receive  a need  for  a service  from  an 
external  source  in  order  to  have  a reason  to  function. 

Our  OBJECTIVE  DEFINITION  block  represents  an  input  to  the  logistics  network.  A 
mission  is  defined,  a job  order  is  constructed,  a telephone  call  is  made-all  these 
represent  the  canminicatlon  of  a nesd  to  our  logistics  network.  That  need  becomes 
a Job  to  the  logistical  network. 

The  indenture  level  of  a job  depends  upon  what  level  within  an  organization  the 
job  is  spawied.  A job  at  one  level  may  became  an  objective  for  the  next  lower  level. 
For  example,  a Ming  Commander's  job  may  be  to  have  his  Wing  prepared  for  certain 
contingencies.  That  job  translates  into  objectives  for  squadrons  within  the  wing 
from  which  jobs  are  defined  and  pei*fonied. 

P * 

It  is  important  to  conceive  our  logistical  network  as  a horizontally  flowing 
entity.  A vertical  hierardqr  implies  levels  of  importance.  Our  network  presents  a 
logical  pattern  to  providing  a service.  No  one  step  is  more  important  than  another. 
Each  level  of  a bureaucratic  hierarchy  has  its  logistical  network  so  a nunier  of  our 
logistic  networks  could  be  placed  on  top  of  one  another  to  represent  logistics  at  many 
levels  within  an  organization's  hierarchy.  As  in  the  example  above,  one  level's  job 
nay  become  a lower  level's  objective. 

Just  as  the  environment  excites  the  logistical  network  by  presenting  a need,  it 
also  detenaines  to  idiat  extent  logistics  fulfills  that  need.  As  the  decision  block 
CONTINUE  shows,  the  envirorment  tells  our  logistical  network  when  to  stop  working  on 
a particular  job. 

Except  for  where  a need  e'-ters  our  network  and  two  other  external  interfaces,  our 
logistical  network  is  essentially  a closed  loop.  Node  2 represents  the  point  to  which 
the  r,etwork  eventually  loops  to.  Should  the  network  not  be  able  to  handle  a feedback 
situation  then  the  situation  may  transfer  from  the  logistics  network  to  the  environ- 
ment via  Node  99. 

( Evaluating  Resource  Requirements  (Figure  2) 

Chapter  II  discussed  the  transformation  of  objectives  into  physical  resources. 

Once  the  resource  requirements  for  a job  are  determined  these  requirements  must  compete 
for  those  resources  within  a limited  resource  arena. 

The  DETERMINE  REQUIREMENTS  PRIORITY  subprogram  is  actually  a queue.  Requirements 
. for  resources  to  accomplish  jobs  enter  a queue  and  are  eventually  satisfied.  However, 

L a priority  algorithm  rather  than  FIFO  of  LIFO  procedures  determines  the  order  in  which 

“ requirements  are  to  be  processed.  For  perhaps  80  percent  (Pareto's  Law)  of  the 

requirements  the  priority  algorithm  can  assign  a nonconfllcting  priority.  However,  the 
/ other  20  percent  involve  conflicts  that  are  beyond  the  priority  algorithm's  scope. 

* Therefore,  in  this  subprogram  management  must  sort  out  requirements  with  apparently 

equal  priorities  and.  Indeed,  prioritize  them. 
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Figure  2.  General  Model 
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In  our  network  the  requirements  priority  beccnes  Job  priority-  When  a require- 
ments package  emerges  from  the  queue  it  is  inmediately  processed  and  as  soon  as  the 
neces^ry  reswrres  bec«  available  they  are  focused  on  the  Job  to  be  accomplished. 

circuit  options  exist  within  the  network  to  temporarily  abort  Jobs 
tte  ^ ^ accomplished  within  a reasona^time.  However, 

are  SKJLdyS  t£^2ul!  ^ * -J®**  ®"“  requirememts  for  that  Job 

poTf»!?Tv‘^T?®*  ^ta  bank  and  sensor  connection  to  the  OETEIMINC  REQUIREMENTS 

?***  priority  decisions  are  not  made  in  a vacuua. 
Worities  are  ba^  on  the  most  current  guidance  available  and  a freling  for 

^ 1nterr«i>tions®may  occur 

aMiySdV%?  «"r1roi»ent  are  not  satisfied  with  the  priority 

* political  override  of  the  priority  decision  shortly 
Actually,  a political  override  may  occur  at  any  point  in  our 
netiwrk.  Hoi#ever  political  overrid^  occur  prinarily  to  change  prio*‘ities 

priorities  have  been  established  and  a Job  moves  forward  in  our  «v«tMi  f« 

Sriis'issfi-Si  ‘•I”"’  z 

all  Of  thf  ^ ‘ require  formal  acquisition  are  reserved.  If  some  or 

the  fltiJl  perform  a job  are  not  available  within  the  system, 

di<r.«IS  process  is  entered.  The  network  describing  this  process  ^ 

Getting  Resources  to  the  Job  (Figure  3) 

P^«T>“e  of  the  REVALIDATE  REQUIREMENTS  block  and  the  HAS  TIC  OBJECTIVE  CHANGED 

?^f”n  'KS  ni?;:  alSSito  ST" 

The  Job  (Figure  3) 

logistics  network's  service  to  a custtxier.  Logistics 
does  not  actually  perform  the  mission  but  jobs  are  necessary  so  that  the  miccinn  ran 
taacctop  Istad.  For  a,a.,le.  losHtIcs  tons  no"f”TbS*l™  itoSST^blTt  ^jlT 
services  it  provides  make  the  launch  of  a bomber  aircraft  possible. 
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.^.i^*-***!*®***®?  earlier.  In  a bureaucratic  hierarchy  our  logistics  network  aav  be 

Em  • “r  "F  ys 

9 resources  togetlier  and  performs  a job  that  in  turn  supports  a mission.  | 

'ti.  ^ perforronce  quality  influences  the  possiblity  of  success  of  a nissfm  ‘ 

i5frinhoS*iniJi5S‘iS^j?o2i^iS?'^‘/2  « 

®f!I!l®5  **  o'^leted  with  the  resources  available  to  it»new  reouirenentc 
network.  Note  that  even  though  a“obl?nIt^ 
additional  requirements  nst  still  compete  for  resources  along  wiJh  other 

s A.', .-  * » ****  ’ ^ ^^1 

-coVeilTJ  ^ ‘ 

subp5iSt.“fiJ;j’l“*i;%‘Sc':s5S  Sd?,f5SnJ1,7»!“*  "* 

_ Acquisition  Process  fFioures  4 end  «;)  - J 

svst^to^unenrl^a  "*”*^^"54®  3MS"are  the  main  resources  used  by  a logistics 

E^!Sl“Lr 

#1^  ♦kf  sources.  These  alternative  sources  can  be  described  in  th^  mntovt 

ilf*®"  required,  the  first  step  is  to  determine  the  source 

Th"  Ss?il  aSlSi^i  ®"«^t<Y«1y.«cquired  in  lijht  of  thi  as^i^Jed  priSuJ. 
the  fSnJwiJg  fable:  ^ considering  tradeoffs  among  the  possible  actions  in 
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Once  the  nethod  of  acquisition  has  been  determined  and  the  amount  of  material  computed, 
a check  is  made  to  determine  if  money  is  available.  If  no  money  Is  needed  or  money 
Is  available  the  material  is  reserved.  If  money  is  needed  but  not  available  a branch 
to  Node  3 Is  made.  The  next  step  (Subroutine  B)  is  procurement  and  is  addressed  in 
another  section.  Once  the  material  has  been  secured  (in  hand)  a check  on  the  system 
Is  made  to  determine  if  all  requirements  have  been  met.  If  so,  the  logistics  process 
enters  the  Distribution  Phase;  if  not,  a recycle  through  the  sane  material  steps  is 
performed  until  all  requirements  have  been  met  in  one  form  or  another. 


Nen.  When  a requirement  for  men  is  levied  on  our  logistics  network  a determination  of 


available  labor  sources  is  made.  Source  options  for  meeting  manpower  requirements 
include,  among  others,  drafting,  recruiting,  contracting,  and  retraining.  Hanpower 
skills  and  nwnbers  are  next  calculated  and  a check  on  manpower  dollar  availability  in 
accomplished.  If  no  money  is  required,  or  if  sufficient  funds  are  available,  the 
manpn^,er  requirements  are  reserved  and  the  acquisition  process  transitions  into  the 
Procurement  Process  subroutine.  If  money  is  required  but  not  available, a branch  to 
Node  3 is  executed.  Once  all  men  are  secured  (on  hand)  a check  is  made  to  ascertain 
that  all  requirements  have  been  met.  If  so,  the  logistics  process  enters  the 
Distribution  Phase;  if  not,  a recycle  through  the  same  manpower  acquisition  process 
takes  place  until  all  requirements  have  been  met  in  one  form  or  another. 

K If  a logistics  system  has  a single  common  denominator, it  Is  money.  The  3Ns 


can  ail  be  expressed  in  terms  of  dollars.  The  annual  budget  is  prepared  in  dollars. 


Occasions  arise  where  money  itself  is  a requirement.  The  000  operates  on  a fiscal 
year  (FY)  basis  and  as  such, the  first  question  to  be  asked  after  a requirement  for 
money  has  been  established  is,  will  the  money  be  available  during  the  current  FY?  An 


affinnative  answer  to  this  question  will  result  in  still  another  interrogative 


statement:  will  it  ($)  be  available  in  time  to  meet  its  requirecaent?  If  so,  an  update 
of  the  dollar  availability  data  bank  is  made  and  the  proper  amount  is  held  in  reserve 
for  the  intended  purpose.  In  case  the  FY  allocation  cannot  cover  the  money  require- 
ment, separate  steps  are  initiated  to  request  the  additional/supplemental  funds  once 
It  is  determined  that  additional/supplemental  funds  will  be  received  in  time  to  meet 
the  requirement.  Once  the  allocation  is  received, the  dollar  availability  data  bank 
• is  updated  and  the  funds  are  reserved  for  their  intended  purpose.  If  funds  are  not 


approved  or  will  not  be  received  in  time, then  a breakout  of  the  acquisition  process 


is  made, leading  back  to  Node  99  for  further  decision-making. 

Procurement  Subroutine.  The  procurement  subroutine  relates  specifically  to  a military  i ) 
environment  as  outlined  in  the  Armed  Service  Procurement  Regulation  (ASPR)  although  ^ 

the  techniques  and  tradeoffs  cited  may  be  used  in  a commercial  setting.  The  first 
step  in  the  procurement  subroutine  is  to  conduct  an  acquisition  analysis  to  establish  ^ 


a basis  from  which  an  appropriate  contract  type  may  be  deteniiined.  There  are  several 


considerations  in  the  analysis  including  the  following: 

(1)  Price  analysis  of  the  industry 

(2)  A cost  estimate  of  the  goods  or  services  of  interest 

(3)  Determination  of  the  urgency  of  the  requirement 

(4)  The  technical  capability  and  financial  responsibility  of  the  contractors 
• (5)  Adequacy  of  the  various  contractors*  accounting  system 

(6)  Other  concurrent  contracts  held  by  the  prospective  contractors. 
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The  next  step  Is  to  determine  If  advertising  is  possible,  I.e. . soliciting  a contract 
froa  a conpetitlve  aarfcet  of  nany  contractors.  This  procureaent  technique  Is  the 
aost  desirable  as  It  results  In  a fira  fixed  contract  uhere  the  contractor  assuaes 
all  risk  of  goods  aanufacture  or  service  provision;  and  under  this  contracting  tech* 
nlque,  a fair  and  reasonable  price  generated  by  coapetitive  forces  In  the  aarket  can 
be  fairly  tall  answered.  The  alternatives  to  a fine  fixed  price  contract,  where  the 
lowes*^  bidder  fs  usually  chosen,  is  negotiation.  Several  contract  types  i^y  result 
froa  negotiation  depending  on  the  aaoiat  of  risk  shared  between  the  govemaent  and 
contractor  and  the  aaount  of  Incentlrizatlon  Induced  In  the  contractor.  Special 
situational  contracting  types  are  also  possible,  e.g..  the  total  package  procureaent 
of  the  CSA.  As  negotiation  Is  an  uncertain  process  In  Itself,  an  additional  check 
Bust  be  aade  to  Insure  that  negotiation  of  prine  contracts  wd  possible  subcontracts/ 
special  agreeaents  and  clauses  has  been  coapleted.  The  final  step  In  the  procureaent 
subroutine  p>1or  to  receipt  and  acceptance  of  the  contracted  for  goods  and  services 
Is  contract  a<ta1n1$trat1on.  Contract  adalnistratlon  can  Involve,  depending  on 
contract  size,  aonitoring  the  contractors  production  process,  siqiplles  accounting, 
quality  control,  cost  accouiting.  and  a aultltude  of  other  contractor/goverraent 
Interfaces.  Necessarily  Node  77  ties  contract  aMnIstratlon  to  Job  quality  control 
to  provide  fee<t»ack  to  the  contractor. 
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Distribution  Process  (Figure  7) 


The  distribution  process  can  be  one  of  the  aost  coapTex  of  all  the  subroutines  due 
to  the  imcertalnty  of  the  economy  and  political  considerations.  There  are  five  basic 
transportation  nodes:/  rail,  highway,  water,  pipeline,  and  air.  The  first  step  In 
the  distribution  process  Is  to  detervrine  what  Is  needed,  where,  and  when  keeping  In 
Bind  that  the  relative  Ixportance  of  each  model  can  be  measured  In  terns  of  alleage. 
traffic  voliBtt.  traffic  revenue,  and  the  nature  of  traffic  coaposltlon.  These  con- 
siderations lead  Into  step  two  which  Is  a tradeoff  analysis  aax)ng  the  various  nodes 
In  view  of  the  essentiality  and  characteristics  of  the  good  or  service  being  trans- 
ported. the  relative  Importance  of  transportation  mode,  and  the  operating  characteristics 
of  the  modes.  Operating  characteristics  which  should  be  used  In  tradeoff  analysis 
Include:  speed,  availability,  dependability,  capability,  and  frequency.  Once  the 
tradeoff  analysis  has  been  completed,  an  additional  step  Is  needed  to  detervilne  and 
sub^uently  acquire  additional  resources  necessary  to  accoapllsb  the  desired  dlstrl- 
Mtlon  of  resources,  e.g.,  a forklift  or  money  for  postage.  Finally,  a decision  aust 
be  aade  regarding  partial  distribution.  I.e.,  should  the  partial  distribution  be  aade 
or  held  for  receipt  of  the  entire  requirement. 


u 


Disposal  Process  (Figure  8) 


Disposal  Involves  a planned  Interface  with  the  environment  to  pass  resources 
back  to  the  environment  after  job  completion.  Sometimes  rejuvinatlon  1$  cost 
effective.  In  this  case  rejuvinatlon  constitutes  a new  requirement  that  demand 
resources  and,  therefore,  a priority.  ^ 


o 


If  rejuvinatlon  Is  not  des1red,used  resources  are  passed  to  the  outside  envlroraent 
by  sale,  decomposition,  mothballing,  or  junking.  Any  of  these  activities  Involve  jobs 
that  mst  be  performed.  In  some  cases. resources,  usually  In  the  fom  of  money,  are 
passed  back  to  our  network  and  routed  to  the  money  pool. 
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There  are  many  insUmces  %#here  the  production  rate  of  an  aircraft 
aiirfraae  nanufacturing  progran  is  changed  due  to  reasons  external  to  the 
production  process.  The  stretchout  of  B-1  bomber  production  and  P-15 
.fighter  production  for  budgetary  reasons  in  1977  are  examples  of  changes 
that  slow  doim  production.  The  change  of  F-4  production  from  12  to  67 
aircraft  per  month  daring  the  Vietnam  emergency  is  an  example  of  a case  . * 
where  production  was  accelerated.  -ci 

These  exogeneous  changes  in  production  rate  are  accompanied  by  a need 
to  know  their  effects  on  cost.  Annual  btidgets  must  be  adjusted.  Contracts 
most  be  renegotiated.  It  follows  that  there  is  a need  for  a procedure  to 
systematically  estimate  the  effect  of  changing  the  production  rate  on  the 
cost  of  producing  aircraft  airframes.  ^ * ..-im 

A primary  element  in  a cost  estimate  is  the  number  of  direct  labor 
hours  required  to  output  the  product.  Many  other  cost  elements  in  the 
indirect  cost  category  are  factored  on  direct  labor.  Therefore,  an 
improvement  in  the  ability  to  estimate  total  cost  would  logically  start, 
with  an  ij^>rovement  in  the  ability  to  estimate  direct  labor  hours.  ^ 

Accordingly,  this  effort  is  focused  on  estimating  the  change  in  direct 
labor  hours  resulting  from  a change  in  aircraft  production  rate.^ 

In  the  paragraphs  that  follow,  other  contributors  to  the  literature 
are  briefly  surveyed  and  a shortcoming  of  learning  curve  techniques  is 
explored.  Then  a theoretical  disctission  of  the  production  rate  is  presented. 
An  approach  for  adding  a production  rate  variable  to  the  learning  curve 
model  is  postulated.  A section  describing  an  empirical  test  of  the  ^ 

approach  follo%#s  and  the  paper  closes  with  a summary  of  findings  and  some 
conclusions.  But  first  it  is  proper  and  ii^rtant  to  look  at  some  of  the 
thoughts  of  other  authors  %rhen  considering  the  effects  of  production 
rate  on  direct  labor  resource  requirements.  ‘ - ■ 

, r ' ) - r:  t 


c. 


^*his  paper  is  extracted  from  the  doctoral  dissertation  of 
Lt  Col  Larry  L.  Smith  (12).  A copy  may  be  obtained  through  the  Defense 
Documentation  Center  under  file  number  ADA0261 12. 
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fro  attempt  to  separata  the  effect  of  production  rate 

«.rT  ^1?  cuBulatlve  production  on  unit  cost.  Be  notes  hommoi- 

^t  there  are  at  least  two  ways  in  ^Aich  the  rate  of  production  can 
influence  unit  1i»>k^  cost  Pirsti  <«-  ....  prOTuccion  can 

machine  set-un  First,  it  ^ affect  the  number  of  hours  of 

"**  charged  to  each  unit.  Second,  it  can  affect  the 
umber  of  subassemblies  employed  in  the  innufacturing  process  this  in 

s:  S'"  «» ■»“»  ssS  toii^ 

unit.  Be  concludes  that  except  for  these  two  effects,  there  is  little 
reason  to  expect  the  unit  hours  for  a 200  unit  peTm^^th^e  L> 
significantly  fewer  than  for  a 30  unit  per  month  case  (2:87). 
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urtber  suggest  that  unit  costs  should  increase  with  increasing  production 
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rates,  •mis  behavior  is  attributed  to  the  fact  that  higher  production 
rates  require  use  of  more  overtiae  and  reliance  on  less  efficient  wrkcrs 
(1:315). 

• 

Gordon  Johnson  describes  an  approach  to  estiaating  direct  labor  * 

requirements  when  production  rate  changes  .occur  on  rocket  motor  production 
lines  (6:25-41)  • He  develops  a model  from  empirical  data  that  incorporates 
both  the  rate  effect  and  the  learning  effect  in  estimating  direct  labor 
hours.  Johnson *8  approach  is  to  regress  direct  labor  hours  per  month  as 
a linear  function  of  production  rate  in  equivalent  units  per  month  and 
as  a pomer  function  of  cumulative  units  produced  as  of  the  end  of  each 
month.  He  reports  mixed  results  on  the  four  test  data  sets.  The  model 
is  somewhat  awlcifard  to  use  because  the  analyst  must  estimate  one  of  the 
parameters. 

Joseph  Qrsini  tests  Johnson's  rocket  motor  model  on  0141A  airfraM 
production  data  to  determine  if  the  model  is  adaptable  to  the  airframe 
production  industry  (11:53-80).  In  the  C-141A  program,  there  is  no  indi- 
cation that  the  production  rate  is  explicitly  changed  in  midprogram  although 
the  data  reflect  that  the  production  rate  gradually  varies  throughout  the 
program.  • . 

Qrsini  reports  that  the  three  variable  model  developed  by  Johnson 
fits  the  C-141A  data  better  than  a standard  learning  curve  model.  He 
concludes  from  this  comparison  that  rate  is  a significant  factor  in 
determining  manufacturing  labor  hours. 

For  further  comparison,  (]rsini  sl90  tests  the  C-141A  data  in  a linear 
form  of  the  multiplicative  model  y • %diere: 


A,  B and  C model  parameters. 

This  multiplicative  model  fits  the  data  better  than  does  the  Johnson  model. 
Qrsini  suggests  that  ”...  the  multiplicative  models  may  be  more  suitable 
for  analysis  than  Johns<m*s  model  because  they  eliminate  the  requirement 
for  estimating  one  parameter  (11-71). 

Fazio  and  Russell  overview  the  entire  airframe  production  cost  esti- 
mating problem  in  terms  of  the  sensitivity  of  each  decision  variable  to 
the  production  rate  (4:1).  Their  objective  is  to  find  the  optisum  rate  of 


y represents  direct  labor  hours  per  month, 

x^  represents  the  production  rate  in  equivalent  units 
per  quarter. 


x^  represents  cx^aulative  units  produced  as  of  the  end 
or  each  month  and 
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production  given  the  decision  variables  of  production  scheduling.  The 
approach  is  analytical  as  contrasted  to  the  statistical  investigations 
of  historical  data  reported  by  Johnson  and  Orsini.  Fazio  and  Russell 
observe  that  the  number  of  parallel  production  lines  or  stations  is  an 
important  determinant  of  direct  labor  hours  per  unit.  They  conclude 
that  the  installation  of  • . . . duplicate  load  centers,  %dilch  may  be 
required  for  higher  rates  of  production,  will  in  fact  reduce  the  overall 
. rate  of  learning  and  thereby  increase  total  manufacturing  hours  for  a 
fixed  buy  (4:66).  They  further  note  that  the  efficiency  of  the  lahi^r  t 
force  varies  by  shift.  Accordingly,  the  direct  labor  hours  required  to 
produce  each  unit  could  also  be  expected  to  vary  with  the  nuiriber  of  shifts 
employed  (4:68). 

“ '3* 

Three  investigators  from  the  Rand  Corporation,  Large,  Hoffmayer  and  ’ ' 
Kbntrovich,  report  on  a search  for  parametric  cost  models  that  tiould 
enable  program  planners  to  project  the  magnitude  of  costs  that  could  be 
expected  at  different  production  rates  (8:1).  They  use  statistical  tech- 
niques to  examine  the  effects  of  airframe  production  rate  and  other 
selected  design  parameters  on  major  cost  elements.  The  investigation  is  Ij 
an  aggregative  ^proach  %ihere  a few  descriptors  from  each  of  many  programs 
are  evaluated  simultaneously  through  multiple  regression  analysis.  The 
purpose  is  to  develop  a general  cost  model  suitable  for  predicting  costs’ 
of  other  programs.  Their  evaloation  shows  that  the  chosen  proxy  for  the 
production  rate  is  of  little  value  in  explaining  the  variation  in  cumula- 
tive labor  hours  among  the  different  programs.  The  investigators  conclude 
that  the  influence  of  production  rate  cannot  be  predicted  with  confidence 
on  the  basis  of  the  an^ysis  performed  (8:50-51). 

* ~ i 

Large,  Hoffinayer  and  Kontrovich  also  evaluate  the  cost  model  proposed 
by  Johnson  for  rocket  motor  direct  cost  estimating  and  later  tested  by 
Orsini  with  airframe  production  data.  Using  data  from  seven  different 
airframe  production  programs,  they  found  that  "In  none  of  the  programs 
did  the  inclusion  of  production  rate  Improve  the  coefficient  of  deter- 
station  (rZ)  by  as  much  as  one  per  cent  over  that  obtained  using  cumula- 
tive quantity  alone”  (8:49). 

Noah  reports  on  a statistical  analysis  of  cost  data  to  discover 
the  effect  of  production  rate  on  airframe  cost  (9:1).  His  research 
includes  the- major  cost  elements  for  two  fighter  airframe  production 
programs,  the  A-7  ami  the  F-4.  Noah  analyzes  all  the  major  elements  of 
eirfraoie  cost  but  his  findings  on  direct  laUoor  hours  are  of  particular 
interest  here.  For  the  two  programs  studied,  he  reports  that  the  following 
relationship  models  the  data  well: 

A B C D . 
y ■ e -Xj  .Xj  .Xj  where: 

y represents  the  average  direct  labor  hours  expended  per  pound 
airframe  produced  for  each  airframe  lot, 

e is  the  base  of  the  natural  system  of  logarithms. 
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X.  represents  the  cunulative  volume  expressed  as  pounds  of 
airframe  produced  through  the  midpoint  of  each  successive 
airframe  lot, 

- w-  • ; 'I-  • * 

x^  represents  the  production  rate  expressed  as  the  average 
pounds  of  airframe  delivered  per  month  for  the  period  spanning 
^ the  first  and  last  delivery  of  the  lot,  • ^ 

represents  the  annual  vol'^sne  of  aircraft  ordered,  expressed 
in  airframe  pounds  and 

A,  B,  C and  D are  model  parameters.  ' ’ ’ ^ ^ . 

Roah  generalizes  the  cost  model  by  averaging  the  estimated  regression 
coefficients  derived  from  the  two  sets  of  cost  data.  Re  suggests  that 
this  generalized  model  can  be  used  to  predict  the  effects  on  cost  of 
changing  the  production  rate  of  the  F-14  airframe  production  program. 

Noah  concludes  that  "Our  findings  suggest  that  delivery  rate  as  a proxy  * 
for  production  rate  has  a significant  effect,  and  an  isqportant  one,  on 
the  production  cost  of  airframes"  (9:41).  ^ - -S 

’ J . j' t* ’’4  fsfTyn  vd 

All  of  the  literature  revie%fed  reflects  a comon  interest  in  the  • 
relationship  bet%reen  production  rate  and  direct  labor  reguirements.  " 

But  the  authors  do  not  agree  on  the  form  or  the  importance  of  that  rela- 
tionship. Some  %frite  of  increasing  unit  cost  with  increasing  rate  while 
others  express  the  opposite  vie%«point.  Some  write  of  no  significant 
effect  of  rate  on  unit  cost  %ihile  others  suggest  that  rate  is  an  iz^x^rtant 
independent  variable.  This  research  tests  the  idea  that  the  production 
rate  .dianges  can  explain  changes  in  direct  labor  requirements.  Further- 
more, it  is  postulated  that  an  increase  in  the  rate  will  cause  a decrease 
in  the  unit  labor  requirement  within  the  relevant  capacity  of  a given  plant. 

A Shortcoming  of  Learning  Curve  Techniques 

Direct  T^anufacturing  labor  hours  in  the  airframe  industry  are  often 
predicted  using  learning  curve  techniques.  These  techniques  are  quanti- 
tative adaptations  of  the  idea  that  individuals  performing  repetitive 
tasks  exhibit  a rate  of  in^rovement  due  to  increased  m£uiual  dexterity. 
Observations  of  complex  airframe  assembly  operations  reveail  that  oumagement 
innovations  such  as  i#ork  simplification,  environment  improvement  and 
engineering  changes  also  contribute  to  the  rate  of  improvement.  It  is 
subsequently  found  that  this  rate  of  improvement  occurs  in  a regular 
pattern  and  can  be  predicted  by  using  a single  model.  One  form  of  the 
model,  the  unit  learning  curve  model,  can  be  expressed  as; 

®1 

y^^  * ♦ 10  ^ where: 
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_ ^ represents  the  direct  iBAnufdcturin9  lebor  hours  required 
to  output  each  pound  of  airfraae  in  lot  i. 


represent#  the  emulative  learning  on  all  airfranes  of  the 
aane  type  through  lot  i. 


B is  a coefficient  that  represents  the  theoretical  nunber  of 
direct  aaniifacturing  labor  hours  required  to  make  the  first 
unit  and  . 


; A 


J 


is  a coefficient  that  reflects  the  rate  of  ii^>roveaent  that 
esists  in  a particular  aanufacturing  environnent. 


represents  the  variation  in  the  dependent  variable  that  is 


not  explained  by  the  model  for  each  observation  i. 


1 denotes  the  observation  nuid>ere 


‘ ' yi\9  r 


nodel  is  more  intuitively  appealing  ««hen  one  observes  that  as 
the  total  quantity  of  units  produced  doubles,  the  cost  per  unit 
*>y  «*«  constant  percentage  (lOtSO).  'ihe  model  is  also  frequently  used 
in  a form  uhere  the  y value  is  expressed  as  the  cumulative  average  hours 
per  unit  rather  than  in  instant  hours  per  unit.  . 


It  is  important  to  note  that  while  the  learning  curve  is  essentially 
a trend  concept,  it  is  not  a time  series  trend  form.  Rather,  the  inde- 
P*®^***t  variable  is  taken  to  be  the  nimiber  of  opportunities  to  learn  while 
the  dependent  variable  is  cost  input  per  unit  of  production  (7:1).  Accord- 
ingly, 500  airframes  produced  at  the  rate  of  50  per  month  would  be  pre- 
dicted by  the  model  to  require  the  same  number  of  direct  manufacturing 
labor  hours  as  500  airframes  produced  at  the  rate  of  ten  per  month  id>en 
the  sane  rate  of  learning  (B  coefficient)  is  assuaed. 


This  lack  of  sensitivity  of  the  leajming  curve  model  to  the  producti-3o 
rate  is  a problem  if  the  rate  is  explicitly  ch^mged  in  midprogram. 
Logically,  direct  labor  requirements  per  airfraae  should  change  as  a result 
of  a forced  change  in  the  production  rate.  The  worker  vho  senses  the 
pressure  of  an  increased  production  rate  should  be  motivated  to  work 
faster  than  the  worker  who  senses  a production  line  slowdown.  Higher 
rates  of  production  should  permit  greater  ««orker  specialization  than  lower 
rates  where  one  worker  would  be  expected  to  accomplish  multiple  tasks. 

Tool  and  tooling  set-iq>  costs  can  be  spread  over  a greater  number  of  units 
at  higher  production  rates.  These  factors  suggest  that  the  production 
rate  should  be  considered  as  a variable  in  the  models  used  to  predict  the 
direct  labor  hours  required  to  manufacture  an  airframe. 
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An  airfraae  production  rate  can  be  defined  in  different  »«ys  for 
different  purposes.  Two  proxies  for  the  production  rate  are  used  in 
this  study.  They  eure  a lot  average  manufacturing  rate  and  a delivery 

“te.  r 

^ L r f • . , f» 

» - I ^ . 

A lot  average  manufacturing  rate  is  constructed  by  dividing  the  niad>er 
of  airframes  in  a lot  by  the  time  required  to  produce  the  lot.  The  lot 
release  date  for  the  first  airframe  in  a lot  and  the  airfroe  acceptance 
date  for  the  last  airframe  in  a lot  define  the  extremes  of  the  lot 
*P9a.  This  form  of  the  production  rate  is  easy  to  construct  and  appears 
to  match  %»all  with  lot  average  labor  hour  expenditures.  Bowever,  the 
averaging  process  obscures  any  learning  or  rate  effect  within  a lot.  Thus 
when  the  lots  are  released  infrequently,  such  as  once  a year,  «-H>  number 
of  observations  is  limited  and  the  data  are  someidiat  distorted. 

A delivery  rate  can  be  developed  by  averaging  monthly  acceptances 
for  each  lot.  If  direct  labor  requirements  are  available  for  air- 
frame, the  actual  monthly  airframe  acc^>tance  rate  c^ul  also  be  used  to 
develop  cases  for  analysis.  Historical  acceptance  data  are  readily 
available  and  delivery  rates  are  easily  forecasted  from  contract  delivery 
schedules.  Thus  the  delivery  rate  is  also  a possible  candidate  as  a 
production  rate  proxy. 


A Cumulative  Production  and  Production  Rate  Cost  Itodel 


• f * 


This  study  Is  based  on  the  assunptioii  that  the  production  rate  affects 
the  quantity  of  direct  labor  hours  required  to  nanufacture  an  airframe. 
Since  cumulative  production  has  been  shorn  to  be  a strong  e3q>lainer  of 
changes  in  the  direct  hours  required  to  produce  an  airframe^  addition  of 
a production  rate  variable  to  the  learning  curve  model  should  explain 
additional  varia.tion  in  the  direct  labor  requirement.  Accordingly,  a 
three  variable  model  is  suggested  as  more  suitable  for  explaining  and 
predicting  variation  in  direct  labor  requirements  than  is  the  cumulative 
learning  model.  The  model  suggested  is 


®0  • »li 


• *2i*^  * where: 


y^  represents  the  unit  average  direcrt  labor  hours  required  to 
output  each  pound  of  airframe  in  lot  i. 


*1  represents  the  cumulative  learning  achieved  on  all  airframes 
ox  the  same  type  through  lot  i, 

*2i  lot  i producrtion  rate  for  all  airframes  of 

the  same  type. 
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represents  the  variation  in  the  dependent  vari€a>le  that  is 
not  explained  by  the  Bodel  for  each  observation  i and 


“o' 


and  are  para 


Bters  in  the  model. 


^ chosen  for  inclusion  in  the  node!  in  the 
‘ * n»»be'  of  reasons.  Other  writers  have  suggested 

**Ti?  predictor  in  this  application.  Multiple  regression 
choice.  Finally,  investigation  of^c»e  ' 
test  data  indicates  that  it  wrlcs  well.  - • • ‘ii» 

,!1"  ^ production  rate  within  the  planned  plant  capacity  " 

shrold  Muse  a demaM  in  the  hours  required  to  manufacture  each  airfraL. 

A rate  decrease  should  produce  the  opposite  reaction.  This  theory  is 
w^rt^  by  tlw  ideas  of  specializing  labor,  prorating  set  up  time  and 
* wrkers  to  produce.  . . v hi  i , 

rnurJm  specializing  labor  to  Increase  efficiency  ' 

^‘***  “***"  ***  direct  production  labor  hours 

SritS  related.-  At  higher  rates  of  production,  more  workers  ' 

^e  ^ed.  ^s  permits  the  foreman  to  assign  each  worker  fewer  tasks 
which  tfe  performed  more  frequently.  Thus,  in  addition  to  the  increased 
proficlen^  which  accrues  due  to  the  leaml.ng  process,  some  of  the  time 
^t  would  oe  required  to  change  from  one  task  to  another  is  saved.  At 
decreased  rates  of  production,  the  opposite  cost  behavior  is  expected  as 

production  line  and  the  remaining  workers  are 
requir«®d  to  perform  additional  tasks. 

. utu^  r j .V  •y.'i  i-  4 *;  - r«  . 

For  production,  the  hours  required  to  set  up  machines 

for  fabricating  airframe  parts  can  be  spread  over  more  airframes  because 
of  tte  accompanying  larger  batch  sires.  This  should  also  contribute  to 
an  inverse  relationship  b<*tween  production  rate  and  labor  requirements. 

i 

r^ly,  it  appears  logical  that  the  worker  who  senses  the  pressure 
^ a high  production  rate  will  be  motivated  to  work  faster  than  the  worker 
* pressed  to  finish  a task  because  the  level  of  effort  is  diminish- 

ing. This  assunqttion  is  si^ported  by  the  "toe-up"  phenomenon  familiar  to 
students  of  the  airframe  production  process.  The  "toe-up"  phenomenon  is 
M increase  in  the  direct  hours  required  to  manufacture  airframes  that  is 
frequently  experienced  as  a production  program  is  concluded  and  the 
production  rate  is  decreased  toward  zero. 

I 

^st  squares  multiple  regression  analysis  is  used  to  examine  histori- 
cal airframe  production  data  in  the  cianulative  production  and  production 
rate  cost  model.  To  facilitate  the  regression  analysis,  the  model  is 
^ansformed  by  extracting  the  common  logarithm  of  each  term.  The  trans- 


formed  model  is  log  y.  * log  B B 
is  now  lineaur  in  each^term.  ® ^ 


log  log  x^i  +'t^. 


The  model 
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nuree  hypotheses  are  tested  in  the  research  effort.  The  first  is 
th^  the  production  rate,  when  expressed  in  the  cuanilative  production  and 
prod^tion  rate  cost  nodel,  can  explain  an  ii«>ortant  part  of  the  variation 
in  ^ to^  direct  labor  requireaent-s  to  build  an  airfraae.  The  second 
IB  tMt  the  Bodel  can  be  eoployed  to  explain  the  variation  in  the  hoars 
re^r^  to  fabricate  and  assemble  the  airfraae.  The  third  is  that  the  . 
■^el  is  suitable  to  predict  the  hours  required  to  produce  alditional 
airfraaes  for  the  cases  examined  in  the  first  two  hypotheses. 


- u 


hnalysis  Approach 


'tf  . 1 •; 


I- 

• '-.I  . . ; . . 

cmnilatlve  production  and  production  rate  cost  model  is  evaluated 
usi^  his^ical  data  sets  from  three  airframe  production  programs:  F-4, 
k:-135  and  P-102.  The  selection  of  programs  examined  is  guided  by  con- 
venience a^  accessibility  of  data.  The  P-4  data  are  the  most  complete  and 
comprehensive  of  the  three  programs.  These  data  are  used  to  test  all  three 
w***'  ***  ^“102  data  appear  to  be  very  accurate  but  are  limited  to 

total  hours.  Accordingly,  hypothesis  two  is  not  evaluated  with  P-102  ■' 

1 vll!*  "C-llSA  data  are  available  in  both  total  hours  and  lower  process 
hours.  ‘ f 

- . >;  • "i*  h7>4. 

procedure  is  to  examine  a total  of  16  data  sets  from  the 
i^ivW^  programs  through  application  of  multiple  revision  analysis, 
^fficientt  are  estimated  from  a data  set  that  tailor  the  proposed  cost 
iMel  to  that  set.  Then  the  predictive  ability  of  that  nodel  is  tested. 
Th«e  is  no  ^ttot  to  develop  a generalized  cost  nodel,  only  a generalized 
approach  to  building  tailored  cost  models.  In  this  sense,  each  data  set 
zrcB  each  program  represents  a unique  population. 


One  way,  to  test  the  predictive  abiUty  of  the  full  model  is  to  succes- 
sively  ronve  a few  of  the  most  recent  cases  from  the  data  set.  One  may 
then  perform  the  regression  analysis  with  the  remaining  data  to  develop 
new  regression  coefficients.  Using  the  nodel  developed  from  the  truncated 
Mta  set,  one  can  forecast  the  points  that  are  removed.  A comparison 
totween  the  forecast  and  the  observed  values  gives  a measure  of  the  pre- 
dictive abiUty  of  the  nodel  tailored  from  the  truncated  data  set.  As 

^rements  of  data  points  are  removed  from  the  original 
^ta  set,  a family  of  estimated  regression  coefficient  sets  is  developed 
for  the  cost  nodel.  Each  of  these  tailored  models  can  be  used  to  predict 
8^  particular  future  data  point.  These  predictions,  compared  to  an 
obse^ed  value,  are  the  basis  for  subjective  evaluation  of  the  ability 
of  the  model  to  forecast. 
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Findings  and  Conclusions 


The  cuBolative  production  and  production  rate  oost  aodel  fitF  the 
individual  data  sets  Mil*  In  each  of  the  sixteen  sets  of  data  the  nodel 
is  found  to  be  appropriate.  Statistical  tests  to  support  this  conclusion 
are  conducted  at  the  0.05  level  of  si^ificance.  More  specifically,  one 
indicator  of  goodness  of  fit  is  the  statistic  for  the  full  nouel.  As 
reflected  in  Table  1,  nhich  sunnarizes  sone  regression  results  for  all 
16  models,  the  R full  statistic  is  respectable  in  each  case. 

The  production  rate  variable  contributes  importantly  to  the  explanatory 
ability  of  the  model.  A statistical  test  of  this  conclusion  is  performed 
and  accepted  at  the  0.05  level  of  significance  for  each  model.  A more 
Intuitively  appealing  test  is  the  iaprovoKnt  in  the  statistic  uhen 
the  pxx)duction  rate  is  added  to  a reduced  (unit  learning  curve)  model.  ^ 
Table  1 lists  the  Rr  full  and  R^  reduced  statistics  side  by  side  for  eacdi 
model.  In  each  case  the  R^  statistic  is  is^oved  by  an  nmniint  that  indi<~ 
cates  the  production  rate  is  a valuable  contributor  to  the  explanatory 
abi  lity  of  the  model.  , ^ 

s ' " i ' 

Within  seme  upper  bcimdarr  related  to  plant  capacity,  production  rate 
and  unit  labor  requirements  meve  im  opposite  directions  with  a cause  and 
effect  relationship.  In  every  model  examined,  the  rate  variable  is  nega- 
tively correlated  with  unit  direct  labor  requirements.  This  negative 
correlation  is  also  reflected  in  the  negative  sign  for  the  coefficient  * 
estimate  in  13  of  the  16  models  (Table  1).  In  three  KC-13SA^models,  the 
combined  effects  of  collinearity  emd  great  relative  strength  of  the  cumula- 
tive production  variable  causes  the  estimate  sign  to  change  to  positive 
in  the  full  model.  ^ . .. 

When  comparing  the  effectiveness  of  a manufacturing  or  delivery  rate 
representative  of  the  production  rate,  the  manufacturing  rate  gives  better 
results  in  all  six  of  the  direct  coiiq>arisons.  But  the  difference  is  not 
great  and  either  proxy  is  an  important  contributor  to  the  explanatory 
power  of  the  model. 

The  model  fits  fabrication  hours  per  pound  data  better  than  assembly 
or  total  data  for  the  F-4  and  KC-135  programs.  This  may  indicate  that  the 
fabrication  labor  requirements  are  more  sensitive  to  production  rate  than 
the  other  two  levels  of  the  process  examined.  Fabrication  and  assembly 
hours  are  hot  evaliiated  for  the  F-102  program. 

This  procedure  does  not  appear  to  be  suited  to  constructing  general 
cost  models  with  coefficients  that  aure  applicable  t ' other  programs.  The 
wide  variation  in  the  coefficients  listed  in  Table  1 for  models  with 
common  production  levels  and  rates  suggests  that  any  averaging  of  coeffi- 
cients would  lead  to  unreliable  results.  The  procedure  appeairs  to  be 
suited  only  for  predicting  additional  production  for  continuing  programs. 
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TABLE  1 
* REORESSION  MODEL  SUMMARY 


Coot 

•Model 

Airframe 

Model 

Casea 

Level 

Rate 

R^ 

full 

R^  re- 
duced 

»0 

®1 

®2 

1 

F-4A-P 

. 57 

Total 

Dei 

0,978 

0.928 

masked 

-0.261 

-0.169 

2 

F-4D-P 

55 

Total 

Manu 

0.973 

0.904 

If 

-0.246 

-O.I83 

3 

F-4B-P 

55 

Total 

Del 

0.966 

0.904 

li 

-0.257  ■ 

-0.161 

4 

F-4B-P 

42 

Total 

Manu 

0.853 

0.585 

'm  ' 

-0.230 

-0.157 

5 

P-4B-P 

42 

Total 

Del 

0.820 

0.585 

m 

-0.229 

-0.136 

6 

P-4B-F 

42 

Fabri 

Manu 

0.889 

0.618 

^ 6.328 

-0.221 

-0.148 

7 

P-4B-P  . 

42 

Fabrl 

Del 

0.851 

0.618  ' 

7.601 

-0.219 

-0.127 

6 

F-4B-P 

42 

Aeeem 

Manu 

0.744 

0.658 

\ 9.016 

-0.279 

-0.112 

9 

F-4B-F 

42 

Asaem 

Del 

0.733 

0.658  ' 

10.400 

-0.278  . 

-0.097 

10 

P-102A 

50 

Total 

Del 

0.979 

0.961  ' 

38.371 

-0.299 

-0.158 

11 

P-102A 

42 

Total 

Del 

0.979 

0.959 

47.290 

-0.344  : 

-0.144 

12 

KC-135A 

96 

Total 

Del 

0.958 

0.971 

13.133 

-0.453  J 

0.164 

13 

KC-135A 

7 

Fabri 

Manu 

0.974 

0.903  . 

0,674 

-0.165  V 

-0.305 

14 

KC-135A 

7 

Fabrl 

Del 

0.971 

0.903  ;; 

1.123 

-0,233 

-0.222 

15 

KC-135A 

7 

Aaaem 

Manu  ' 

0.994 

0.964  f 

13.338  • 

-0.608 

0.361 

16 

KC-135A 

7 

Asaem 

Del  ' 

0.992 

0.964  ; 

. 7.303 

-0.527 

0.263 

rate  variable  stabilizes  and  improves  the  predictive  ability  of 
the  cost  model  for  the  F-4  and  P-102  program  data.  This  improvement  is 
particularly  marked  for  the  nine  P-4  models  analyzed.  The  predictive  accu- 
racy of  the  models  developed  from  the  P-4  data  sets  make  them  attractive 
alternatives  for  estimating  labor  requirements  for  additional  production. 
Although  the  model  fit  for  the  P-102A  data  is  excellent,  it  does  not 
forecast  the  prediction  target  very  well.  Tests  for  predictive  ability 
improvement  are  either  inconclusive  or  impractical  for  the  KC-135  program 

i"  • < ' I « . I I • ti  • 

■ ( . I 

The  procedure  is  wll  suited  to  forecasting  the  direct  labor  require- 
ments foe  additional  production  lota.  The  steps  for  such  a forecast  are 
reviewed  here.  After  the  coefficients  are  estimated  axtd  the  cost  model 
tailored  to  a particular  program,  one  first  aaist  decide  on  the  accept- 
ability of  the  model.  If  the  model  fits,  its  predictive  ability  should 
^ tested  to  gain  some  measure  of  confidence  in  a forecast.  Assuadng  that 
the  results  of  this  test  are  acceptable,  one  must  calculate  the  cumulative 
production  plot  point  and  the  delivery  or  manufacturing  rate  for  the  * 

unknown  production  lot  or  lots.  Then  using  the  tailored  cost  model,  the  , 
unknown  direct  labor  requirement  for  the  new  lot  can  be  predicted. 

ISic  oonclusions  described  above  are  necessarily  limited  to  the  data 
sets  examined.  But  there  is  great  teaqptation  to  generalize  the  results 
to  other  airframe  productioom  In  particular^  the  notion  that  a productioo 
rate  variable  correlates  negatively  and  importantly  with  direct  labor 
requirements  has  much  application  in  the  airframe  industry  if  universally 
true. 
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